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ABSTRACT 

This paper describes progress to date in the development of a syitan for 
representing various fonts of real-world knowledge. The knowledge is 
stored in the fora of & net of sample parallel processing elsnents, Which 
allow certain types of deduction and set-intersection to be performed very 
quickly and easily. It is claimed that this approach offers deficit* 
advantages for recognition and many other dati-ACCOSSiUp tasks. 

Suggestions are included for the Application of this system as a tool in 
vision* natural “language processing,, speech recognition, ansi other problem 
doflplns. 


This report describes research done at the Artificial Intelligence 
Laboratory of the Massachusetts Institute of Technology, support for the 
laboratory's artificial intelligence research is provided an part by the 
Advanced Research Projects Agency Of the Department of Defense under Office 
of Naval Research contract N0D01^-7S"O0i&^. 
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1, About This Taper 


In my thesis proposal [4], I described ft pi AH for irtp 1 e me n t In (J th« 
frame systems of Hiny [12] and applying these to various recognition 
tasks, late last Spring (1574) this research took ft surprising turn.. One 
of the many schemes J had tried tor representing class-inclusion (IS-A1 
hierarchies'-ft' scheme using Quillian-Iike nets of parallel processing 
0 1 nm en ts - - pr 0ve d to have so many interesting properties ahd appltcations 
that the study of this system has become the central focus of my research.* 
Among Other things, this new representational system eliminates the neotl 
for a network of demons for suggesting recognition hypotheses, anti It makes 
the checking of hypotheses much easier. 

After a month or twp oT ploying with the new system {to convince 
myself that It really Wft£ What it seened to te), I decided that it was time 
to write up my preliminary results. For a variety of reasons, it has taken 
me almost A year to produce this write-up. Ihe set of ideas that make up 
the system Liecaree very large very quickly. And Simply wnuld not stand Still 
long enough, to he written about coherently. Always there was one Bore 
loose end. to tie up, one more example to Work out, one more idea to fit in. 
Anti the very process of writing about the systen, on the few occasions when 
I actually got started, seemed to trigger major collapses and 
reorganisations, All this has led to rep id progress In the research 
itself, but not to a paper. 

At long last, 1 have been able to get a sort of snapshot of the 
current state of my research down on paper. The old problems reraaiu :■ there 
are many loose ends and inconsistencies, especially in the newer ideas 
presented toward the end of the paper, and there hoi been ft considerable 
evolution in ray thinking even as this was heing written, 1 hope that All 
of this will not prove to he too confusing. A rtOCC coherent presentation 
will he possible later, after things have had a chance to settle. 


Z. QvgruU-w 


On its simplest level the new descriptive system consists uf a 
large s*t or nodes that can bs marked in various ways. These represent the 
objects, classes, and other conceptual entities that the system knows 
about. Connecting these nodes, and representing the re)atinnships between 
the node-concepts, are several types of imis. Ruling o-ver the whole 
network is toe central processor (CPU), a serial computer that broadcasts 
commands to the nodes and links via a comron party-tin-e bus, Markers are 
passed ami prepaflfltod from nods to node along the lint paths under the 
strict control of the CPU, Since the markers con travel along many 
branches in parallel, the system can mark very large trees or nodes in only 
a few steps, 

So far, this sounds like QuiUlan's network system. The difference 
is that by mating thus type of network with a set of fairly rigid 
organisational principles—a hierarchy of IS-A relations, for example—l 
have ho on able to establish much morE precise control over the flow of 
markers through UiC net, guillian's system might typically be asked to 
find all of the associations bet-neon LQKti and HAN; rty system would 
typically be commanded to nark every man whose grandfather 1 ! mistress had a 
long nese- (This is not to S*y that Quill Ian's system could not irt 
priori pic follow complex paths or mine frEe-associPta — the difference is 
mainly one of emphasis,) 

3f> much for the inspiration. The porS-piration has been shed in the 
attempt to find exactly the right set Of organizational principles —the 
right way to- represent various sorts of knowledge—using these nodes and 
links, This task is flPt complete 95 yet, but enough of the systCO has 
taken shape tp suggest that the finished product will help to soIve (or 
Circumvent) a nunber of long-Stflndiiig AT problems. A brief summary of some 
of tli? system's wore interesting features: 

-Type or I3-A hierarchies are efficiently implemented. Members of a class 
inherit all of the characteristic properties of that class and of its 
super-classes, merely by 'plugging in - , . Access to these properties is fast 
■ftbd does POt require pre-computation, redundant Storage, or lengthy 
deduction at access time,. This CirCiUPventS the problem, that so plagued 
tharnUk (among many others) of bow many "obvious n inferences should he 
made when a new fact is received, dud how nuch should he left undone until 
the neo-d arises* 

-The system cap handle many orthogonal IS-A hierhFcheS. tangled together tc 
any desired extent. 

-It is very easy in this system to find th£ intersection of two or more 
classes. These classes cati be cxrlitit groupings represented in the TS-A 
hierarchy, or they can be implicit groupings based upon sohe cc-mnon 
property of the members, Thus, nne can feed the system a list of 
properties observed in a 'sample, and have it return a list of ail known 


objects or classes that exhibit such propErt ies. There is more to 
recognition thee this--the list of properties mny be ihCOmplete, imprecise, 
exceptional, or in error--but this mec.hn.nism ran carry r.uch of the load. 

Id particular, it performs much more elegantly the task ! had previously 
assigned t& the Swam of "suggestion demons" ; the discovery of hypotheses 
worthy of further consideration. 

-When an individual Is Assigned (perhaps hypothetically) to a class, it 
inherits the descriptive properties of that class, Of ton some fentnres of 
this description will cts^Pr with features already present in the object t 
either directly observed or inherited from some previous assignment r The 
system CAP detect such clashes very quickly and easily,, and can then begin 
looking for excuses, or whatever. This quick clash-detectloil is obviously 
useful in testing recognition hypotheses. 

-The same clash-detection mechanism, can be used to enforce res trf cftpns 
Upon the possible occupants of the various "slots" in a frame, a relation, 
or a verb, -Such a restriction tan be very general ("animate object*) or 
very specific ("left-handed Bulgarian tuba player").. Any attempt tn make 
ap assignment that violates such a restriction Is lmmedlately detected by 
the system, (One can, of course, force the assignment to occur anyway. ) 
This nechani&m would appear to have many uses, especially in linguistics, 

-Each piece of information in the system exists in some rojrlext,, and is 
only visible if the context is active. This allows the system to maintain 
many distinct WO rid-node Is, Information not in active contexts is 
completely out of the way and docs not slow down the accessing machinery, 

-The contexts themselves form a tangled hierarchy with different levels of 
generality. Onu can thus be operating in a very general context with a lot 
of information available or in a very specific context with much less 
potential for afr.li i gu i ty and con fusion, The system focuses its attention by 
nuviitg up, down, and around on this hierarchy. This would appear to be a 
□uch smoother mechanism than, say, switching between a few distinct mini- 
worlds, 

“t'hile I don 1 ! want to press this claim too strongly, this system or 
something similar would appear to be a plausible metaphor for Jiunafl- 
intelligence, The elements seem rather neuron-like, and because of the 
parallel ism they -could he rather si oh {mil) 1 seconds) - The system seems to 
mesh neatly with the requirements oT a linguistic systun, And, to me it 
least, it rights the things that ought to be easy turn out to be 

easy. (I’ll point out some examples of this ss I go along,) I hope in the 

coming months to See what light the psychologist* can shed on this issue, 
and th gather together any hard data that 1 can find, pro or con. 

Is the special parallel hardware really necessary to reap these 

rewards? I think SO. The essential step in all of these operations is the 
murk i hr) of all the nodes in soma tree or other. These trees are seldom 
more than ten gr so links deep, but they are bushy, containing thousands OT 


fivun millions of nodes. The parallel scheme narks such trees In tine 
prop or 11 mini to the- diipthj while the time to mark Serially grows as the 
total mrmher of nodes. 3t is my system's ability to make liberal use of 
tree-hiiu-kjug that gives it any special properties it might hflve. A 
reconniCion does Indeed have fi sequential component r blit it is a spgisenee 
Of parallel tree-mark Wins, Of course, we can simulate such a system on a 
serial machine for testing purposes, aspue tally if the total size of the 
knowledge base- is kept reasonably small. 


3 r The iSyrrilDl-rlA[ipinu Problem 

Suppose [ tell you that a certain animal--let 1 s call Mm Clyde--is 
an cl^jihant, Vou accept this simple assertion and file it away with n.0 
apparent display of mental effort, And yet, as a result of this simple 
transact ion , you suddenly appear to know a great deal about Clyde, it 1 say 
that Clyde climbs trees or plays the Diane or lives in a teacup, you will 
immediately begin to doubt my credibility. Somehow, "elephant" is serving 
as more than a mere label here; it is, in Seme sense, 0 whole package of 
properties and relafc ion ships, and that package can be delivered by means nf 
a single I5-A statement.. 

En principle, such hc-havior can be achieved through the use of tone 
form of deduction, inch fact Is a separate entity, and new facts urc 
produced by knocking together two old ones. Thus, If we have "All 
elephants have wrinkles" and "Clyde Is aa elephant", we have the right to 
deduce that Clyde has wrinkles. In One form or another, this has been the 
'Standard At approach, 

Eut having the right to deduce some fact is not the sane as having 
the Job done, fluch ingenuity has been devoted to the search for fast 
deductive mechanisms, but the problem remains intractably combinatorial, 

And w.hun Is all this work to be done* ]f we are to detect the Obvious 
clashes between new facts and old, some deduction must bo done at once upon 
• receiving now information. But we clearly cannot afford to deduce tflf of 
the consequences oT the new fact-^to do so would take a. very long tine and 
would hope Acs sly clog our memory with useless trivia, Thors sec-ms to be no 
good way of deciding how far this process should go. And can we really 
believe that all of this frantic deduction goes OH while the listener 
believgs that he is simply accepting a single, straightforward fact? Thfl 
fastest machines bog down when faced with a few hundred facts, Cap 
mill lsecnnd ■ speed neurons succeed with millions of facts? 

I could not help 'feeling that senathiug was ttissing horo--tbat the 
discrete-fact deductive approach was never going to solve the elephant 
problem. Does this natter? Well, consider where Al has been successful 
and! where it has, so far, failed. The triunphs have been in areas like 
calculus where the symbols being dealt with carry very little semantic 
bflggiagCr Micro there arc rooms to traverse, or tables full of blocks, or 
missionaries and cannibals, these are abstracted and stylized^ all but a 
very few properties ar« eliminated- Such a pitiful number or facts can 
indeed be■handled on an individual basis. 

■ ‘ . 

But what of the real world, full Of shoes and ships and sealing 
wax, whore elephants have not only size, shape, nnd color, hut also 
Wrinkles, blood, flies, and an insatiable lust for peanuts? The list is 
practically endless, and that is exactly the point, If the phrase "Common 
sense" means anything at ail, it oust certainly include an awareness of 
those ^fringe" properties. In any given situation, one of these 
insignificant details may bo of pivotal importance. I think that our 


critics- - Dreyfus, for instance-ire right in claiming that Al has made 
little real progress Crt this, front. 

]f fact'by “-fact deduction is inadequate.. Is there perhaps 5 one way 
□ f dealing with whale groups of facts at once 1. This possibility first 
occurred: to ne while ] was playing with the multiple data-base contexts of 
CMIWIVER, which allow whole sets of facts and demons to be made visible or 
invisible by a single declaration, [ also had in n 1 nd Ah image derived, 
from the "pure' and "impure' code in a computer System: The elephant 
description should riot he copied -Mid raodiFied for each elephant; rather, it 
should be kept "purg* and individual elephants should bo described by 
"plugging in" to this description and adding a small package of local 
assignments and mod if lea Cions to thO general randel. This image was 
strongly rein forced and influenced by Ninsky's from? theory* Eventually, 
all of this led to the- packet ays to*! described in my thesis proposal* 

But packets woro not the right Answer either. It is all well and 
good to iflakG the elephant description appear and disappear, but the real 
prPblem is to turn ail of the ELEPHANT properties into CLYDE properties* 1 
call this the siP7ii(iI-J9tfppiiq,y prohJem. For a while, I thought that i could 
leave the owner of a property unspecified, but this led to terrible 
problems of ambiguity, especially where the property was really a relation 
betwoon two nore-or-les* frqutl entities, as In FATHER-OF, I had been 
playing with various parallel hardrtare schemes for implementing packets 
efficiently, and l decided to try something sinilar for symbol mapping, 

IIIiA was meant to be an exploratory step, but the result was SO successful 
iu SO many ways that I stayed with it. (Incidentally, I later realized 
that packet activation was just a special case of symbol napping and tbat 
tt required the Same sort of fflechanisras for success,.) 


4. Nodes end Links 


We Hill no* consider in some detail the nature nf these nodes and 
links, And how they can he Used Eo solvg thE symbo 1 -tuppin 5 problem, This 
did net material J ze all at oncer l am skipping over about a half-dozen 
icilemediate steps in the system's evolution. 

The various concepts in the system—for no* we can limit these to 
individual objects like CLVDE and groupings. like F.I.E PHANT--are represented 
by nudes. These are relatively Simple hardware Units, Each, one contains a 
permanently-assigned unique soriiri: auCTfcer an-d a half-dostm or so 
intiopen (I out flip-flops tailed mar*#/ 1 flits. 1 mentioned earlier that the 
nodes could ho marked in various distinct ways; this is done by setting one 
or more of the darker bits in a given Node unit. These bits are designated 
by letters j thus, we speak of A node as being "narked: with an A-marker 11 if 
Its A bit J,s on, 

Each df the nodes is connected via a corunnn party-line bus to the 
CPU, from which it gets its Orders. The CPU can specify any bode by serial 
number and can order that node to alter or report its murker status, ttbre 
orten. a command will he broadcast to any and all nodes containing sone 
particular marker or combination of markers. Sometimes the CPU will Just 
want to know if any such nodes exists sometimes It will want these modes to 
change their marker state In some way; sonotlmes it will want the selected 
nodes to report their serial numbers over the bus, If many nodes try to 
report at once, they are queued up in order of their serial numbers (or 
perhaps .\p order of their position along the bus) and they report in one- 
fey-one,■ 

These nodes are analogous to USP atoms in .several respects. The 
actual word "elephant" is not a part of the ELEPHANT mode, but U attached 
to it at a property, {The details Of this will be discussed later,) 

Instead, the node we will call "ELEPHAfJT d represents the COfltept of 
elephant and is known to the system by its semantically sterile serial 
number. A few nodes may have some Special mMnJmg to the CPU (like MIL in 
LloP) or some direct association with the raw output symbols of the I/O 
system, but most derive their meaning from the way they arc connected with 
Other nodes. 

These connections are represented by hardware fin* units running 
between the nodes in question, for now, let us think of those links as' 
coming in many different flavors, corresponding to the different passible 
relations in the system,. Thus, to represent ¥ Clyde is an elephant" we 
simply connect an IS-A link from the CLYDE node to the ELEPHANT node. 
Similarly, we might hav f $ n Ms link from ROCKEFELLER to STANDARD-OIL a 
FATHER-OF link from AD-RAHAN to ISAAC, or a COLOR link from ELEPHANT to 
CiRAV. These links have a direction, in the Sense that a relation like OWNS 
dr FATHER-OF is not commutative, but markers can he sent across thea in 
oiLher direction. {We will :ater see a way of getting along with only a 
singR, acre complicated kind of link.) 


TJvflSC link units are con 5 ]dorably mnnu than mere libeled wires, 

Eacb Is it hardware dniE that is connected to the party line bus, and each 
Is able to carry out simple commands received from the CPU. These hardware 
units are connected ta the nodes that they are supposed to be linking by 
dedicated private 1ines--actual wires that are not shared with any other 
link unit. Over these private lines the link units dan sense and alter the 
marker status oF the attached nodes. When I speak of a rt Link n I will 
usually he referring to the hardware link unit find to the attached private 
lines ns a single entity, 

The coma lids sent from the CPU to- the link units have two parts- 
first r a specification of which links are to respond; then, a Statement of 
what those selected links are to Jo, UsusaHy, a whale sot of links will 
l»i' ic j ec«.,Gd at Qncfi'i. on the bAsis of th#ir type -and til# marker status qF 
the attached nodes. A comae ed miglU, for instance* he addressed to all is- 
A links whoso In Caning node (the X in "X I5-A V") Is marked with hit A and 
whose outgoing node [f ip the above emirtple.} is not so marked. The CPU can 
sense Whether any links claim tncrnb e rship ip this group. The comanij 
follows: in this case, it might specify that each Of the selected Lints 
should mark bit A of its outgoing node. Because of the private line 
connections, all of the selected Links can perform this operation 
simultaneously. The net effect is to propagate each of the A markers In 
the system across any adjacent oirEjotrrff IB-A links, all in a single machine 
cycle. 


How, if we mark CLYDE with bit Af and We then repeat the above 
■operation until nothing more happens, *e will have placed an A narkar on 
every node that represents a class of which Clyde is a member, either 
directly or by transitivity of l$-A, If, instead,' we nark MAMMAL initially 
and proberate the markers across IS-A links ih the opposite direction, we 
will have narked every node that represents a kind of MAMMA!, and every 
individual HAMMAL- Regardless of the number of nodes marked in such art 
operation, the time-required is proportional only to the number Of links 
forming the longest branch of the tree. It Is very hard to tltihk of a 
reasonable chain of IS-A relations longer than ten or fifteen linkj--try 
it! What all this means is that, given this hardware, we can use tree- 
markihg as an operation with no more trepidation than a LISP user would 
feel.in USiog a C0-.M& or a SfcTg, Even if the tree is, say, the tree of all 
physical objects, Even if the elements of the system operate us slowly as 
say, neurons. 


And this, I claim, breaks the back of the symbol-mapping problem, 
yhen we toll the system that Clyde is an elephant, it simply creates dh 15- 
A link from CUBE to ELEPHANT. Ur the process, It Checks for clashes-- 
inore about this later.} ELEPHANT, of course, is linked to many nore 
general categories: NAHm, AN INAL, QLlAERLTPED, HERBIVORE, PHYSICAL-OBJECT, 
and 30 on, £aeh of these Classes has its own sat of characteristic 
properties, represented by property links attached to their nodes, and all 
Ot these properties are to be inherited by GLVDE. Uow r if we ask the 
system for, say, Clyde's color, it won’t hhow which node to look at, but It 


doesn't matter! In a single Sweep, it marks all the nodes superior to 
CLYDE; then, it asks any CQLQR-dF links attached to crap or the marked nodes 
t.n mark (with a different bit} the associated color node.; finally, it asks 
the marked color to report itself to the CPU, Thus, It become4 unimportant 
whether a property is attached directly to CLYDE, or Is really attached 
j QBewhe r e far up a chain of [fl-fl Jinks, T.n ere Is much to he added to this 
p it turn--CSC opt ion S , for ins tone e— but this is the key idea. 

The opposite marking process—from a class to its sub-classes and 
members--has its Oses as well. Suppose we have an OWNS link from ONG- 
ARTHIIR to F,J(fAt 1 RC'R, his famous magic sword. -Suppose that we want to know 
if any kings own any weapons. We sinply mark, with bit A evpry ICING, mark 
with hit E every WEAPON. and look for an OWNS link between those two sets. 
Out would pop the ARTIIliPJEJiCALlEUR link, along with any others that the 
system might happen to know. To mark all weapon-ownmg kings wo would park 
all weapons (bit A), then all their owners (bit B]+ and finally all kings 
(bit C), Any node narked with both & and C It a winner, 

iloforo wo go on to the mors complex issues that arise in trying tc 
use those nodes and links properly, perhaps it would toe worthwhlie to 
briefly consider whether we can, tn fact, build a system like this using 
existing technology. The problem, 1 believe, is not with the node and link 
units themselves —those are indeod expensive, but not ridiculously so given 
LSI technology—but rather with the private lines connecting them. As the 
system learns new things, It will meed to create new connections and 
occasionally destroy old ones (though some form nf cancellation might 
substitute for 1 ink-destruction}, This link-alteration cart afford to be 
rather slow, since it corresponds to a lonj-term memory alteration e there 
are tricks we can use to represent a (lotted number of vuick, temporary 
□ Iterations until they are no longer needed or can be wired-in permanently. 

There would appear to be two possibilities: either we rust have 
sore device or process that can connect (or grow} a wire directly between a 
node and a link terminal, or we need some kind of crossbar switch. Imagine 
a set of vires - -one from each node—running north/south In a piano. Above 
this is another Plano, made of link-wires running east/west. Wherever two 
wires cross, a connection can be made or broken; this is called the cro^S 
point. It would Obviously bo too expensive to put an actual switch at each 
such point, but perhaps there is a cheaper way of making and breaking 
contact at such points —sane sort of electrochemical gao that turns locally 
conductive when appropriately zapped and stays that way iintil it Is somehow 
unzapped. it’ would help somewhat to group nodes and links into richly 
connected neighborhoods, with relatively few connections running to Other 
neighborhoods, Put this night be more trouble than it is worth. 

What I'm trying tp say, then, is that it would be Tflr tow UXponsivt 
to implement this system on 4 large .scale with currently available 
hardware, but that the dev#lopmemt of a much ctoaapsr implementation 
technology is not inconceivable. Jn the reantime, the system is woII 
Enough defined to serve US as a precise metaphor, and it can he simulated 


far testing pruposes, >1 Of course,. in the simulation, tree-marking will 
again be proportional to the nuiibar of nodes in the tree, but wo- will be 
9t)le to tell how long the hardware w&Ul-d have taien.) 


5 . The IS-A Hierarchy 

Tba backbone of the descriptive system--tbs rigid structure which 
keeps everything else from collapsing intfl a heap --15 the network of IS-A 
links. These links tie all of the nodes in the system isito ft single 
tangled hierarchy of groups, subgroups, and individual nemhers. With 
perhaps ft few exceptions (to he noted Tatar) ftvary node is connected by at 
least one 3S-A link to some more general category that includes it; ft very 
concept has an identity. Eventual ly. alj of these upward-moving IS-A 
chains conv^rye Upon the system’s most general node, named TH1 WO, which 
includes everything (e-very-THTO) else, 

Before proceeding let me make explicit a convention that I hove 
already used once or twice, and will use a lot more in the remainder of 
this paper: IS-A links will be spoken of ftS pointing Upward, Ths MAMMAL 
node is oJjoyft ELEPHANT and ieiom AMT HAL, The more general categories ore 
the superiors or ancestors of the Less general enes, THING is St the tap 
of the tree-, CLVDB and other individuals at? ftt the bottom. Links 
representing nther properties run more-or-less sideways. This ■will ift ve ft 
lot of words in the Ions run. 5tlth m luagt is possible r of course, 
boceuse the IS-A links do not form loops, (Exception: w* may £fl,y A IS J A & 
and B TS-A A if we want to ihdieftte that both nodes represent the same 
concept. This is useful in certftin perverse cases.) 

Probably the mg at inpartant aspect of the 15-A tree is the way 
properties are inherited, Consider ft rode like COW. (One grows weary of 
elephants after a while.) Properties and relations attached to the CPW 
node itseir, or to eiitf flotfe dSfftCtilr (Jftove it tfl tile fi-A tree, are meant 
to apply to aid. cows (though particular individuals can cancel particular 
inherited properties). Properties hung from nodes below CW apply only to 
so.vie cows. In Figure l p for instance, we see that o!£ cows are warm¬ 
blooded and have udders (barring exceptions, that is), hut that only stuns 
trows are black-and-white, while others are brown. There are not, to the 
systenT-s knowledge, any purple cows, hut this is not ruled out. If asked 
what color cows are, the system would first mark. Upward from CflW, but would 
find no COLOR links there. It wculd then mark downward from COH+ find 
several COLOR links, and report that cows in general have mo particular 
color, but that some cows are brown and some black-and-white. 

It is important in choosing marking and accessing Strategies, 
always to respect the transitivity of the IS-A links. If COW IS-A MANUAL 
and MAKHaL IS-A ANlnAL, then COW 15 -A ANIMAL, whether or not there is a 
link explicitly saying so, This effect is achieved if we are careful 
always to lflt upward or downward marker sweeps run to completion! wg should 
be very suspicious of Any accessing strategy that sends markers some 
particular nuaber of stupa up or down the tree, 

A corollary of this transitivity is that we are always free to 
split an ]S-A link, for instance tn add the node UHGl’LATE between COW and 
MAfiflAL, Once w* have added COW IE-A UNGULATE and UNGULATE IS-A MAMMAL, the 
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original I5-A link frn m COW to HflPIAL becomes rtfdvn dan % - The system can 
reclaim this link or leave it in place, whichever is easier, As you can 
see* it nakes sense to talk abnut cine nodfi being above Of below another>, pr 
between two ethers, or heing off on some- unrelated branch of the IS--A tree, 
but it makes very little sense to talK. about distances up end down the 
tree, slnca thc&ti cam be measured along various paths and changed at will . 

The 1S-A network is a tenglecf hierarchy., By that 1 mean simply 
tliftt a given node nay have more than one immediate superior, but this 
simple fact Can have ime interesting global effects, In the nore familiar 
ncui-tflhg i«d hierarchy, a node nay have many H5-A links fanning out below 
it r but only one link to a superior, This convention leads to downward- 
branching trees like the one in Figure Za, in which nP two branches ever 
recombine ohee they have separated, figure £fr shows the tangled hierarchy* 
in which the branches do tangle back together at various Lower nodes. 

The difference Is more dramatic if we look at the tree from the 
point of view of some typical node‘-call it X>-in the middle- of the tree 
(figure 5), In either case, the descendants of X fan out into a profusion 
of stlbcl asses and members, hut look inn upward from .X it is a different 
Story, In the untangled hierarchy,. X has only a single it rand of 
superiors, while with tangles the tree of X's superiors can fan out to 
considerable -diinsnsinns before it starts converging hack towards the THlffG 
node. X IS thus id a position, td inherit a very large and diverse set of 
properties, h'ichout some sort -of parallel accessing system this could be 
an embarrassment of riches, hut with the nodes and links described earlier* 
a large tree can be marked and accessed as easily as a single Strand of 
nodes, 

It night seem that there is some danger of runaway markars finding 
their way through the tangles and marking everything, tie can avoid this by 
taking care to murk Ofllj; upward or only downward from a node, but never 
both in a single sweep. (RenEODer that there are no significant loops -0-f 
15-A relations In the system.) If, for some reason, we should want to mark, 
hat/i the ancestors and descendants of a node, we Should do this with two 
separate sweeps, and then convert the markings uf une set to natch the 
other, j 
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6. Exclusive Sets and Clashes 


the need Tor tangles in the I5-A hienrchy arises Tron the fact 
that most group nodes can be divided into sub-group5 in many different 
wayj, according to more-or-less orthogonal sets of featurus. The PETCStW 
node, for example, can be divided nr the basis of age, sex, occupation, 
rac&, height, ami SO on, Whilie we would not normally have occasion to 
tangle together the results of any Single 5 -plit”-rtflL£ and FEMALE, Tor 
ExamplC^we nay very well want to recombine- the results of di/Jerent 
splits, ns when we recontoine MALE Uhd CHILD to pet BOT. Figure A Shows a 
part Of the tree that hangs below PERSON, The existence of a rccombined, 
node like BOV not only gives US a way of including an individual in both of 
the superior categories with a single 1S-A Unk; It also gives us a plhtO 
to hang any properties that are characteristic of the CMihUaUoB of HALE 
and CHILD r 

The arcs in figure A indicate various cicItiiiPe Jets among the 
subgroups of PERSON, An individual or group may have an ,15-A Unk to one 
member of eurrJi of these groups, but only one. Any attempt to Lin In¬ 
directly or indirectly--to a second member of an exclusive set should be 
detected by the system as one forn of cEcjft- 

The actual clash may occur Far dp the tree from the offending HOtto* 
To take a rather far-fetched example, let us try to imagine a marsupial 
mushroom.. These two classes do not directly clash with each other, but if 
we trace the I5-A chains upward, we find ourselves marking both PLANT and 
All [HAL, which are part oF ah exclusive set under LIVING “ TH1ND-5Wo might 
be able to make sense of this by treating ono class or the Other as a 
metaphor or analogy--fiei'haps we have a fungus that protects its spores in n 
pouCh--&Ut wo are not free to accept this description literally, as we 
would if the object wore, say, a marsupial herbivore (no clash). 

He would, of course,, like to have some quick way of detecting these 
clashes. Even if this only catches gross absurdities, that Is at least a 
step toward some sort bf comnoh souse. The trick Is tu create two distinct 
flavors of nodes, OXCIUJl£VS and a da- exclusive (or regular) nodes. These 
types chin dither he inherently different or permanently marked with an 
exclusion bit or labeled with an appropriate link, whichever is easiest in 
a given implementation. A node like PERSON, which encemjiasses several 
exclusive sets, becomes a regular node With several exclusive nodes bung 
below it (figure 5), 

A clash is detected whenever marking upward from a single node 
causes markers to pass through an exclusive rode From two different 
incoming IS-A links. Suppose we want to add a new [£-A link above a node X 
that already HAS some connection to one or more higher nodes. First, hb 
mark with bit A all of the suiiorlors of J according to the old links. 

Then, we send bit E propagating up the rrris Link In such a way that when a 
E-marker enters an A-marked node, it deposits a B but goes no farther. We 
then ask any exclusive nodes with both A and 0 marks to report In. IF 
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them are auy, th k sc are the clashes, 


Any specific suggestions that the system night have for dealing 
with a particular clash can be hung from the exclusive node representing 
the clash, hut usually wo will just resort to general methods for 
reconciliation, Depending nn the evidence hr have for assorting each or 
the Clashing links,, we night sinply reject hue nr treat it as a metaphor. 
Sometimes WO can turn off the alarm and ignore the clash, but this usually 
just leads to more clashes as we try to reconcile the properties of the 
incompatible classes. (Blood or sap? Cellulose cell walls or not?) 

Sometimes, as with the auglepa or the sex of ap earthworm, wo will have to 

create a new set to represent the individual in question and add this to 
the sot of possibilities under the exclusive node; this most often occurs 
when there is a twilight tone between the categories or sane sort of hybrid 
o>f thorn, but it can represent a comp Lately new possibility. 

The mechanism so far described catches only clashes between 
expitcii categories to which an object is assigned; clashes between the 
properties- of an object can he more difficult. If wg say that a person h.fl£ 
a Iona hosb, and later assign him to a class that implies the possession of 

a short nose, we night not notice the discrepancy which would be obvious if 

Iony-ho sod people and short-posed people were considered as distinct 
classes, ¥e car.'t flood narfcers out to all of the parts and properties of 
a description and look for clashes, Since the loflfi of "long npse" would 
Clash improperly with the siiort of "short finger", or whatever. 


1 have not yet really attacked this problem, since it depends on 
certain details of the basic link format that have net yet been worked 0 Ut T 
For now, just let me say that clashes between an object's superinr classes 
□re trivial to find; clashes betwEen plaiat properties of an object 
(herbivorous-carnivorous, largo-small) appear to he detectable by a single 
swpep ui well; clashes between the properties 0 / parts would appear to take 
a separate sweep for each part, I don't think this is too inconsistent 
with human capabilities- dac final note; any particularly important part- 
properties Can become whole-pro per ties, as when a person with large muscles 
becomes a muscular person or a person with yellow hair becomes a blonde. 

We will see how such tram format ions -occur in a later section - 


Tho exclusive nodes give us a way of indicating what an object is 
pot . If we are told that Clyde is pot a naranal, we simply create an 
exclusive set containing CLTUE and nANHAL, (See figure 6.) Any subsequent 
Attempt to connect CLlfPE to MA1NAL,, or to some sub-MANUAL. such as ELEPHANT, 
sounds, the alarm. 


Andee Rubin [i5] has suggested, in a similar context, the use of 
complete: exclusive sets, and Rick Grossman [EJ has built an entire 
descriptive system using such sets. These are Just like regular exclusive 
sets, except that it is mandatory for an individual to fit into exactly one 
of the sub-classes in the set. This convention comes into play when wb 
have ruled out all but cne of the possibilities in the sat; If I tell you 


that CLYDE is a vertcbrate, but that he is nat a fish* reptile* bird* ®r 
anpltibian, you can be reursonffJhitf sure that ha is a mammal. 

The problem, as you can iee, is that there very few classes nf 
anything {mitside a-f mathematics ]• that cad be divider) so cleanly that wc 
are sure that we Injvii listed uil the possibilities. Even in SO crystalline 
a dlChPtcmy as sex we find scene isolated cases of hermaphroditism, XXV 
chromosome sets, and so OH. Ehit "reasonably sure 1- Is usually good enough, 
aort the process of elimination does seett to play a part in our reasoning, 
especially wac-ri; the allegedly complete exclusive set has only two or three 
members, Sherlock Holmes, for one, used this sort of reasoning disite 
of tee* and with excellent results, 

it is a simple enough matter to create a third type of node to 
represent these complete sets and to treat them as exclusive nodes for 
Clashing purposes- What I have not yet found-Ohd do not really expect to 
find—IS A fast parallel way of detecting when an object has closed off all 
but one of the possibilities of some complete set somewhere* so that we can 
rush in and plug the object into the remaining possibility. This lacH does 
not worry mu too much: it seens to me thit when hunans reason by 
elimination, they proceed laboriously, case-by-case, sometimes making a 
list Of counting on their fingers. 1(1 view gf this* it would be surprising 
if the machine wero too good at such reasoning. We will see a bit more 
About this later* when we consider how the machine digests new information^ 







7, Equivalent Forms and Winston-learning 

Beforfi I go on to introduce more machinery, I would like to clear 
np -a few remaining points about the I5-A iiiornrchy, and especially about 
how it grows and changes as new nodes and properties are added. For now 1 
will continue the practice, introduced earlier, of representing each 
property or relation--COLOR, OWNS, FATHER-Of--by Its own type of lint. In 
a later section we will see what these lints re-gily loot like. 

The first point, to consider Is that the transitivity of the is-A 
relations makes it possible to represent certain relitionships lit a variety 
ot ways. In Figitfi 7*. hp see that wtien a number 1 of nfldfrs claact membership 
in -some SOt or superior groups, WO Can create a new class representing tbo 
intersection of these superior groups and Simply link the individual nodes 
to this, {The picture seems to say it better than the words.) 

Such transformations are not mandatory. All of the nodes ip the 
left-handi d la gran are hooked up to the proper superiors, and the System can 
gel along this way indtfinilely. But, especially when many individuals 
belong to a set with ttany superiors in ccnmop, making the transfoUmatton 
can save a large number of links at the Cost of only a Single new node, 

Thus, while there is no particular urgency about it, the system can benefit 
from spending soma c-F its spare time looking for situations that can be 
profitably transformed in this way. ThR eanct threshold of profitability 
in such cases depends upon the details of the implementation: the relative 
cost of nodes and links, whether old links can he recycled, the cost of new 
connections, and so on. 

Figure R shows a similar equivalency based or properties rather 
than group membership. Where a number of individuals have some common 
property or group of properties, it is possible to make such properties 
characteristic of a group. Individuals, instead of having to claim each of 
tlfCSB properties directly, sinply join the group by using On IS -A link , I 
will re;?: to this jrocess--creaEinD a group-node and moving an appropriate 
set of properties from Lower nodes Up to the group--by the term 
'generalization" acid to the new nods as a 'generalization node". 

These seIT-reorganisation processes, I believe, Jire responsible for 

l^v ^cTp nw? n ° r the va - rr " 1:ic,rit 7 °f tiie network's frid-Level nodes, A node 
_iie j.c.RpaiAlfT is created as soon os somenpe has seen two or three 
individual elephants, to serve as a place Co attach the very large number 
04 properties that these individuals share. The simplest way to establish 
such a node Is to commandeer the node that represents the first or most 
typical member of the group to be the new group node, leaving its 
properties intact, and creating a new subordinate node for the individual, 
iropertlts which, as more elephants are added, are found to be 
idiosyncratic rather than general are passed back down the tree to the new 
individual iumJhj , 

L,iese -generallist!on classes are more than a mere device for saving 
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4 Efrw links l combined with a suitable recognition Strategy, they play an 
enormous role in organizing our perceptions. Consider how well the human 
species would have farad U the rerpeity of each Individual animal held to 
. 0 *Y ua 1 ,n dapendejitly. By the tine some particular saber-toothed 
tiger has taken a bi?e of you, it is probably too lath. It is a far batter 
strategy to assign a new Individual to some group on the hasls of the 
raadLly-nhservable information and ip assume that the rest of the groups 
properties hold as well r e™ though these properties have not been 
■observed directly. ]J any of those assumed properties have been explicitly 
contradicted by observation,, of course, the system must either change the 
i^ejn tif ication nr make- ^xplicil note of the exceptional properties.. 


£d we See that hy properly creating and using these BAfleralizatiorv 
nodes, we can make & few observations do the work of many, in much the same 
way that wc got CLVD£ IS-A ELEPFMNT to do the Work of many Statements. The 
SUtcess of this approach depends upon the domain. Of course. In domains 
where thi- properties are rather ■Obvious and form coherent and widely 
separated groupings--animals for example--lt will be very successful. In 
domains where the prpperties are themselves hard to recognize and where th* 
groups S)iur together ihd contain many exceptions, things will be much more 
difficult. So while everyone agrees that ELEPHANT is a valid and useful 
class, there is much less agreement about SCHIZ0PHHEW1C or LIBLRM {the 
Philosophy, UPE the party). Still, I am convinced that most of the 
categories needed for common SGhss reasoning are of the Coherent, ELEPHANT 
type. Ovorzijalous use of generalisation and An unwillingness to recognise 
exceptions are dangerous, as well] these are the sources of bigntry and 
racism, along with many other fonts of too-rigid behavior. The'system must 
try to find the best balance. 


^ concept ELF-PHAMT is not some Innate ideal category [though 
PhT .5 3 CAL-OBJECT may be); nor is it some precise mathematical predicate 
[ able to mate successfully with the Prototype Standard Elephant, kept in a 
vault in France' ), though progress il* genetics may reduce it to that; 
ELEFFlANT is merely a shorthand for a bunch Of pro per list that soon to otctir 
together Often enougih fnr the Shorthand to be useful. 

This way of looking at concept-nodes is an CKtShsion nf fand is 
directly descended frc*i) Winston's description-learning system (23]L J 
have emphasized the process by which generalization nodes are created in 
the first place, as the system Itself searches out parts of the pet that 
can be profitably reorganized, Winston studied Che process by which a 
useful generalization can be passed from a teschcr to a learner, once it 
exists somewhere in the culture. EnStead of having t& search blindly for 
places to put generalization nodes, the learner is told, for Instance, that 
»nru ra ^n^^^ rartt structures are all to be considered members Of the dess 
AftCFI, By showing the learner a carefully chGSGh sequence Of members and 
non-nemburs of tha class, the teacher is able tn control with great 

precision which individual properties will bt moved up to the new general 
node. ■ 


It h-fiS been observed many times that ths language itself carries a 
substantial load of Winston-instruction in Its collection of words, Not 
every node has a nj^, but every name lias a node. By Observing- what 
objects the hams is applied to, the learner can gradually ascertain the 
boundaries of the Intended class, and it is then usually a simple matter to 
ripduce which properties should be novad up. Random conversation can thus 
serve as a train ing sequence, though not a Very efficient une r horn 
difficult concepts are better conveyed by nore efficient means: explicit 
definitions, as when we state that a mamnnl is a warm-blooded vertebrate 
with hair and so on, This can be converted npre or less directly to i 
piece of network. 

In any event, there seems to be reason to hope that if wq can find 
the right set of self"reorganization procedures, along the lines discussed 
above, wo should be able la get the system to do most of the work in 
cresting its own knowledge net. Much of the rest cdn perhaps be taught 
‘instead of programmed, though there will be some minimum of "innate' 
structure upon which the rest is built. This Is an old drtam, of course, 
but It seens mere plausible in the context of this type oT system. I have 
riot even begun to work out the details of the reorganization strategy. 


A. Digestion and the "Every" Problem 


Qno *orc loose: end to tie before me go on. to bigger things; Al the 
logicians in the audience have 00 doubt already noticed, the systbPI 
described so far has no good way qf representing such statements as "All 
purple mushrooms are poisonous." We can state that ail mushroons are 
poisonous by simply hfteglruJ & FDSMNOL’5 property on or above the TlUSN'RTOri 
node. We can state that purple poisonous mushrooms exist hy creating a 
Itode holow HUijHFiDgrt and hanging PURPLE and PdlSCNOUS from this, (It nay 
also be necessary to somehow declare this set non-empty.) Bet the fact 
that this class exists does not mean that we are required or oven allowed 
to move every purple mushroom down into it-, any more than we are allowed to 
move civery gray antraa] down into ELEPHANT, ] call this situation the 
"every* probleia., The solution is not too difficult, We label certain 
nodes as being mcfl-dotury and divide the pro parties of those nodes lrrto two 
classes: eaSedtiul and incidental properties. (We will see later a 
variety of ways of attaching those labels.} The I'WLE-flUSKfctKJfl class is 0 
mandatory subclass of MUSHROOM; It has PURPLE as an essential property and 
POISONOUS ftS all incidental, (See Figure 9.) 

How, whenever we add a nw Bt*U>er to nus-URUCR or add a new property 
to some mush roan, we check the various naniatary subclasses of MUSHROOM 
(and or its superiors) to see whether the Individual fits into any of them, 
To fit, it must explicitly satisfy all of The essential properties of the 
mandatory node; in our example, the individual mist be PUEPLE, Note that 
merely failing to clash with PURPLE Is not good unehflh, hut being sono kind 
of purple is. Once we have found a mandatory subclass which fits an item, 
we nest move that iten down into that subclass. The individual then 
inherits the incidental properties of its new class--in this case 
PQISOUOHS, 

Since the properties of a class can be inherited as well as being 
attached directly, we must also allow the mandatory nodes tn claim 
essential and incidental class membership. Figure lb illustrates this. If 
any individual claims nerabership in ail of the essential superior classes 
of a mandatary node—in this case, if it is both a NUSHROOft and a PURPLE- 
OBJECT--then we mave it down and let it enjoy membership in the incidental 
( FQIEDKOUS-P LAM ) Classes as well. The tun systems can, of course, be 
combined: to fit into n mandatory class an individual must show all of the 
essential properties and belong ta all uf the essential super-classes. 

Tills scanning process, initiated by the addition of new information 
(links) to the system, is an example of whut 1 cull tfijestfoir. The clash- 
checking process and the search for comulete exclusive sets with only one 
regaining possibility are also digestive processes. These digestive 
processes fill the sane ecological niche as the IF-ADDED demons of other 
systems, but they are more limited in power: there are only 1 few types of 
digestive processes, and only a single way of representing each. To ne, 
this seems like flfl advantageous limitation. Once in a great while the 
system may need to change nr augment its central pro gran for digestion 
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control to acconodala 5om- new process, but it is not continually having to 
writs new demon programs or to arbitrate inter-demon disputes r suppose, 

" absence of sufficient hard exper 3 ento, i t is a matter of tflite* but 
[ find trie man l pu lul ion qf explicit date structures to be a much leas, 
fearsome process then the automated production of debugged programs. 

Unfortunately, the digestive process is both tedious and op an’ 
end^d. Tedious, because the parallel tree-marking process cunnot carry 
micti of the load. Some phases of the operation--the marking of all 
possibly-re levant mandatory nodes, for instance--can be done in parallel, 
blit ultimately the system must sift through possibilities on an individual 
basis. (1 think so, anyway.J flpen- ended, because the ts&ential properties 
may t Hems elves have- to be deduced from other things, ad ginrsiH njini fcuw. 

Hie non-nembership of an ilen ii\ some class under a complete e^clusivn node 
taay also require some effort to establish. 

Clearly, vhnt the system mast da is to perform as nuch digestion a$ 
±l ha£ time i!ir < an(i then prnceej to other things. This nouns that some 
poteOtially-daducible consequences of the new link may go undeduced, but 
completeness is not the highest mental v i r the - - human .5 seen to get along 
well enough without it. When the system is hurried it will overlook a 
great many consequences; when it can afford to be mot# contemplative it 
wi.I overlook much less. If a particularly promising digestion must bn 
abandoned, the system can mark it somehow and come back later, when it has 
some spare time, Afid, of course, the system Can cane hack on the basis of 
need—locking for the necessary essential properties when it wants to 
establish Some incidental property; for instance, or investigating semu 
particular area that is suddenly of interest, 

AU of this digestion business is just a forn of logical (induction, 
and not a very efficient form at that, I have taken sane pui„ 4 to include 
because it does seem to play a role in human thought, and because 
qui Ulan's -networks have so often been criticized as being unable to handle 
gwftritillers and Che like. As wo have seen, such things C&ti be integrated 
into the present systen without undue strain, 

Bui the application of such precise logical statements (flil purple 
mushroom?) seem to me to belong to the domain of abstract sequential 
Winking, rather than to the kind Of intuitive, seemingly instantaneous 
thought that characterizes, &ay, the recognition of an elephant. This is 
why ] um not particularly bothered about ray inability to find 3 fast 
parallel way of carrying out the digestive process— why, in fact, t bava 
not really looked very hard for such a method. ! Just don't believe that 
Speed in this area Is necessary for human-level intelligence , 

Perhaps all of this will be Clearer ir Viewed from 0 slightly 
■d:frerent angle. Uhat w E really have are two different, ^hough compatible 
families of intermediate nodes created by two distinct processes. Spno are 
created by generalization frem below, as we saw in the previous Section. 
Ihese nodes refiott the empirical Observation that pertain sets of 


I 


properties appear together in many individuals. F„LErHAPJT is such ft node. 
Individuals are placed in the ELEPHANT group not on the; basis of any orre 
property, but because they fit in that group better than they fit anywhere 
else, end because they do not clash with the ELEPHANT requirements, Any 
set of properties Mill do, as long as they art sufficient to rule out the 
Other competing tJasses; the rest of Che ILIPKAWT properties are then 
inherltod , 

This process of moving a sample down Jntp a class pit the basis of 
whatever properties are readily available is what I have been calling 
rpcognit inn and, as we will see, it can be quite fast. The price paid for 
this Speed is some residual uncertainty that wo are really ifl the right 
place. l»'e don't really have a, decisive test for ELEPHANT, but we can trade 
time for greater certainty! as we gather nore and nore properties that fit 
the ELEPHANT description, it becomes increasingly uniihely that we ate 
dealing with some other animal, 

The other nodes, the mandatory ones, are created not by grouping 
together lowly individuals, bat by splitting apart higher nodes on tba 
basis of some particular property, Once JIL'EHRQOH has heen divided into 
subgroups Oh, the basis of color, we can look for incidental properties 
characteristic of the members of these groups--irl f the membersThis 
process can more legitimately jive rise to sweeping generalities than can ■ 
upward-grouping. The mere fact Chat we have found a lot of poisonous 
purple mushrooms does not give us the right to say that all purple 
mushrooms are poisonous, b»lt IT we have carefully divided all known 
mushrooms by color and have found all the purple oats to be poisonous, we 
can at Least state ow rule for AeiwrT mushrooiis. If we are prepared to 
believe that khDWn mushrooms are all that we Hill encounter, we have merely 
to set up a me chantsn for applying this rule, mechanically and inexorably, 

The real difference, then, is in hew the two systems handle the 
inevitable uncertainty that arises when we try to deduce a property not 
directly Observed, merely because the world has always boon consistent in 
the past. The upward gonoralisetion merely summarizes past experiences; 
the act df faith occurs when the recognition pro-gram assigns a nBW member 
to this class on the basis Of only a few of the necessary properties, 
assuming that the rest will be present. In the downward Splitting process, 
however,, the risk ts taken In formulating the rule ltself--in attempting tp 
derive d Universal truth from some collection of observations. If we 
cheese- to accent this rule, we can then proceed with absolute certainty !by 
logical deduction {or digestion). Ironically, this certainty makes the 
recognition task harder, not easier, The deductive systera ts brittle,. and 
since the slightest error can lead to vast absurdities, the digestion must 
proceed very carefully, Th-fi generalizoCiou'recogrvitio-n process is much 
more forgiving and, 1 believe, plays the central role in the intultivc- 
lovgi thought that so far has eluded US. 


9. In Search Of Cite Rissing Link 


5o much Tor amateur epistemology—it 1 & time to gel back to nuts and 
holts. £peci Fically, it's time to Coma Id er what we really want to use in 
place of our rtrf hoc nenitgerie of link species. Clearly, wo nsust find a way 
of representing all, possible Ulttr-noda relations with a singlE type of 
Link {or at most a very small number of varieties). Occam's razor aside, 
this is a matter of practical necessity: the number of relations in use is 
huge and over-growing, and the inventory problem would be on emeus. 

This inventory problen is solved easily enough by the use of a 
single, standard type of hardware link with a register into which is 
written the name of the relation it represents--FAIH£K-DF or whatever. 
Alternatively, we could make our link units with three distinct private- 
line terminals: two running to the (we node* being linked, as before, and 
the third running off to a node representing the relation. This latter 
form is preferable, since the relation Is Itself represented by a node and 
can thus take its proper plate in the is-A tree, acquire properties, and so 
□n. Ue might, for instance, want to note that FATHER-OF is a kind of 
HELATIVE-OF, 

.1 

This is all right as far as It goes, bat it is not enough, A 
relation is more than just a magic word whose invocation takes ns from one 
node to another--* relation must also have a ^cosing. By its very 
existence, a statement like "Jf is the father of V" tells US somathlilfl about 
X; that he is an adult male animal. To say that HARV is the father of V 
would generate a clash, since we know that the name Mary Is typically 
reserved for females, Ve also know a bit about ¥: that he or she is 
considerably younger than X and is of the same species. 

So each relation supplies a set of re2es or siots into which tha 
itens being related must Tit, and to which they become connected by LS-A 
links (perhaps inplicit ones) when the relation is statEd. These rules m*y 
be very specific classes or very general ones. The WtlGHT-OF relation, for 
instance., has one role Chat can only be filled by a certain kind of unit“ 
srwno number of grams, pounds, or tons—*o<i another role that can be filled 
by any physical abject. The latter category is very large, but not 
universal: ideas will clash, as will substances. Iron, you see, does not 
have weight—it has density; a piece of iron a an object and will not 
clash in this role. 

The mearlng of a relationship involvet more, even, than the 
imposition of roles upon the rulataes. IT we use FATHER-Of in a strictly 
human context, wo find that it conjures up a whole constellation of other 
relations, rules, and even stereotyped actions. 'We find that both X and V 
Are parts of a structure called a family, which has slots for an adult 
female £ K 1 s spouse, Y'S mother) and for numerous other people, all 
connected by a dense network of relations, W c find, in nost cultures, that 
X is supposed to support Y while Y is young, and that Y is supposed to act 
respectfully toward X, And so on. 


The problem, then, Is to .Tifid a representation for links that glve£ 
US access to as much of this meaning as possible without doing a lot of 
deduct inn „ Just as wp wanted to simply plug into the ELEPHANT no-do and 
Inherit all of its properties, we now wnnt to simply hook up X FATHER-QF Y 
and get all the rest for free. {This is not exactly something for 
nothing--the hardware, ytni will recall, is the rather costly secret 
ingredient , ] Of course, wc can riot let oor zeal for efficiency load us 
into representations in which certain things cannot be expressed at all* 
though HB will continue to concentrate the systen's efficiency where it 
seems to he most heeded. 

The battle is far fron over. The mechanism presented from here oh 
are simply ny current best guesses, subject to change. Still, 1 don' t 
think they ere too fir from the ultimate goal, in spirit if not in detail. 


10 , The Three-Pronged DF-Link 


Quite early in the system's development 1 hit upon a lint 
configuration that seemed to did fairly well all of the tasks 1 had in mind 
for it, 1 call this Structure the 3-pronsred QF-link. With ft few 
flmhsll ishnents, this single link type has fceen used in almost all of my 
ttlinking about descriptions and their uses. Other formats have proven to 
be preferable far particular purposes, but none has shown a comparable 
generality. An added beauty of the GF-Link is that it is practically a 
one-to-one mapping of the way property relations are normally expressed In 
English, The T-pron<fed OF-link will he described in this section and will 
be used in most of the examples in subsequent sections. 

Lately, however. J have had some second thoughts about the OF-link, 
1 hftVfc turned rny attention from Static descriptions oF nbjects, parts, 
and properties to pore complex systems of frames and varyingcantexts, the 
OF-link has become increasingly awkward tg use, It is Still adequate, hut 
I have begun to feel that a link based more on the context-smbeontext 
relationship—"in" rather than "on —would he preferable. Perhaps, top, I 
have been tap fanatlcni in seeking a single univarsai link-type, rather 
than a. Taw types, I will return to some of these possibilities later on, 
but for now the QF-Link will serve us. E have mentioned these doubts 
merely to Indicate how far ail of this is from being chiseled in granite. 

Figure 11a shows the basic OF-link. The relation It represents is 
usually stated as *A is the B of C or, if h Is not unique in playing this 
role, as "A is c 1 of C*. This format can bE used for properties, 
relations, and for subdescriptigns or parts. The rest of Figure 11 
Illustrates this: "GREEN is the COLOR gf (every) FROG*; "ABRAHAM is the 
FATHER Of ISAACS "a TRUNK is the ND5E Of an ILErHAM", 

Wow, an important point: £gr/i of these 3-pronged links curries ah 
implicit 1S-A link from A to 1. Thus (IREEW 15-A COLOR, ABRAHAN IS-A 
FATHER, and a TRUNK IS-A NOSE. Properties and group membership- are passed 
down these links just as they would be over a normal IS-A link. In fact, 
we can now eliminate the I5-A link as a separate entity: it is merely a 3- 
prouged!-OF-link with the Gf-proOg LeTt unused. So the OF-link both States 
a property reLation and lacks the piuyer flodr A into tne role B, If the 
player clashes with ttlft requirements of the role, this will be detected by 
the mechanism described earlier' thus. Our suspicion of the statement *flAR,Y 
is the FATHER of CLYDE". 

The variability of the little words--ft, an, the, every--ln the "'A 
is the B of C" formula seems to bother somE people. These words do not 
reflect any dl[Terence in the link Itself, but are rather signals to the 
listener about the context and the nature of the things being linked. 

"The" Signals uniqueness, while "a/an* does not: thus, we have ~tde father 
of Isaac" but "a terminal of resistor ftl.' individuals--Isaac, for 
OX ampie--net no article at all. [We will sen lator how such individual 
nodes are labeled.) To speak of the color Of u frog is ambiguous: we Dean 
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L-verif frog, and t-p Avoid cosnfusipn wo Had better say that, lest the 
listener assure He mean sow particular, hut unspecified, frog, Despite 
the wording, the links iu Figure ]| Ares all the same and are used In the 
same nay. 

These links allow us to create vertical layers of description, as 
shoe'll in Figure 12, a fragneitt of the description of an electronic network, 
A TERM INAL is a PART of an ELECTRONIC CfflHPOWEWT; a TRAN-51STQR is a kind of 
ELECTRQN1C-CMRDNENT and an EMITTER is one of its TERMINALS; 71 is a 
particular TRANSISTOR And El is its EMITTER, T1 inherits the properties of 
the COMPONENT and TRANSISTOR classes; El inherits the properties or 
TERMINAL and EMITTER, Lfe see here,, for instance, that T1 is made of 
COFFER, because all teminals are. 

in the last section I suggested that a relation inposes a role on 
ftotfc of the related nodes- We have s->s e n how the OF-link casts tho player 
nqrle into a role, but what about the node At the end of the OF-prnng, which 
I call thE oirirer? 

There are two answers to this. First, note that if I say to you 
"El is the EMITTER or Tl," you know at cince that Tl is a TRANSISTOR. 

Looking at the EMITTER node, wp can see that it is defined as a part--B 
TERM INAL--of a transistor, and of nothing else. 50 if we--or the systec-- 
are told that Tl has an emitter, we are perfectly justified in assuming 
that Tl is a transistor, arid can -create an IE-A link saying SO, Tills is A 
relatively trivial piece of digestion, 

.Things are more difficult if we are told that "VI is a WHEEL of X," 
All sorts of things have wheels: cars, bicycles, even clocks, 5o if we 
look above the WHEEL node we find links with OF-prong s running off to 
dozens Of other nodes. X might be any one of these. All we ran do, in 
lieu of further In forma t ion, is to assign X to a class general enough to 
contain all the po£sibilities--MAN-MAffE-0(kf£CT 1 perhaps. If, later* we 
learn more about X or more about the kind of nfoeel Vl is, we can then aovt 
X down to a more specific category. Trrifllfli wheels, for instance, appear 
only on bicycles. 

The second mechanism is more direct. Certain roles ennp in 
reeiorocirf pairs: husband-wife, parent--of fspring, own or-property, etc. 
These paired nodes are explicitly md together by a mechanism f will 
explain later. Now, whenever the Systen assigns a player to Ohe role in a. 
pair--"A! is the PARENT of Y*, for instance--It lnust iruied lately create 
another link assigning the owner to the reciprocal rol9- , -ih this case, "Y 
is the QFF5 PRI-NCb of A", Each of thesg roles, of Course, carries its load 
of properties and represents a potential clash. This use of two links 
aaems rather wasteful, but It gets the .job done neatly, Note that if we 
know the 5CX of either the parent or the offspring, we would pove tPiB 
individual id question down iptn one of the- mandatory sub-classes based on 
sex; SON or DAUGHTER, MOTHER or FATHER.. Note, too, that the reciprocal 
pair o-f roles nay not be immediate superiors, but night be somewhcr-B up the 
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tree. Still, the search takes only one sweep. 

j-nn-rn. Tu?f nf thiS explalns h< ™ m are tn represent a relation like 
GREATER-TOAH, "bich does not fit into the OF format. Such things 
represented by a more Trans-like structure which I will explain several 
sections farther on, after more machinery has been developed. 
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II. Accessing Formulas 


if I wflnt to direct, the system's attention to one of its internal 
nodes--perhaps so that ] can tell it some new fact about this node--I hove 
tD find some way of specifying which node I moon. I do not have access to 
toe node's internal serial nunfoer, The mode might or might not hove an 
unamtfignous none that is known to both me and to the machine. If no such 

name exists, hpw am I to point out the node in question? 

The answer, of course, is to construct fl phrase that describes the 
location of this node in tarns of other nodes that can be specified hy 

narta. Returning to Figure 12, I might Specify the node El by the phrase 

the emitter of Tl\ rather tharr by its name. Either fora is completely 
Tin ambigu oils, and I indicate that I intend no ambiguity by my use of the 
word "the", I might instead speeiry *a ternlnal of Tl" Which would refer 
to El, Bl p or Cl. If I spoke of ’£Ae terminal of Tl* the system wuu.Ld 
become unhappy: the word "the" signals that IE should find exactly one 
node, but instead it finds three, It would then have to ask which terminal 
I mean. 

I call such node-spneifying phrases Accessing /nrmulirs , Some other 
examples would be *the length of the hO-se of hax*£ grandfather's mistress* 
or "the mouse that murdered the cat that guarded the cheese In the houso 
that Jack, built*. The production and decoding of such formulas Is Cute of 
the major tasks of any natural-language systfln, and is relatively 
straightforward in an DF-link system. In this section we will be concerned 
only with phrases made from static relations] verbs like "murdered" and 
"built" wall hove to be deferred untii after the discussion of context 
mechanisms. 

Let us, first consider how to find and mart 'the terminals of j | w in 
the structure Of Figure 1Z. First, wn will assum* a trivial mapping fron 
names like TER. ft INAL and Tl to the associated concept nodes. (We will Sea- 
Plow to use context to Separate Multiple word-npauings in a later section.} 
Our fsrst impulse might be to send a command to all links Of the form M ? 
is a TERMINAL of Tl’ to nark their plnynr-hodes. Tbit will not work, for 
there are no such links. The TERMINAL link is attached not to Tl, but to a 
superior node. 

A better plan is to mark all of Tl's superiors, and theh mark 
□cross any TERMINAL links connected to any of these. Rut this is still not 
right: we have marked FITTER, RASE, and COLLECTOR—not thu desired El, 

El, end Cl, We have to move the markers back down to the level of Tl, but 
wc can 1 1 just sweep downward from the nodes we h 4 v 0 already reached. That 
would mark the terminals of etrery transistor, not just Tl. The proper 
p, ocedure is to move the markers downward nver only those link! whose OF- 
prongs are attached to nodes that we marked on. the way up. This will bring 
th:e markers hack to the proper level--fly farther--and keep them from 
wandering off to unrelated term 1 upIs. The markers go up the left sids of 
the ladder, across, and down the right Side. This Is called a I p defer 


jfiT(?rp. 


Sometimes the markers will not make it all the way back, duwrtIf 
we ask for the 'lAlERiAL af El" we will get COPPER, Up could create' fl, new 
node below COPPER to represent "th* COPPER of £]■ but there IS no point in 
this uni ess wo jiflve something to say ahnut il--that It Is very pure for 
example. 


This raises an interesting painty m general, nodes are only 
created as they are needed, J know that every person has a liver, but if I 
have ten thousand person-nodes in i»y lietid, I da not necessarily have teh 
thousand liver-nodes. I only have liver nodes for those livers that I 
happoti to know something about, And yet, if you toll ne that Clyde's liver 
IS purple, it is a Siraple matter to create tho (lode LI and: assert (LI is 
the LIVER of CLYDE) and (PURPLE is the COLOR of LI), Actually, the U nod* 
would not be given the name H U M or any other n*n*. it would just be 
"Clyde 1 s 11 vgf , " 

Long chains of those relations should in principle be no harder to 
follow. If we have 'the length of the nose of the mistress of the 
grandfather Of Max" we simply use one luddar sweep to get IYoei HAX to the 
node representing his grandfather, another to get to the mistress, and so 
on- I *un assuming thflt the system can convert "Kax'i grandfather" to ‘"the 
grandfather of Kan" without too ranch trouble, 

tfftat if mr ask the system to mark "all men whose grandfather's 
distresses havo lung noses"? First we mark all nen. Then, as above, we 
send out a chain of markers from each man tn his grandfather, then to the 
mistress, tn her hose, and fihally to the nose’s Length, These steps can 
bo done lor all men in parallel. Some of thirst chains will end on the node 
LONG and Others on the nods SHORT. (Ue are assuming quantum no&6S here.) 
Many of the chains will die along the way: nme grandfathers, probably had 
no .mistresses--at least ones that the system knows ahout, then simply 
place a new marker an LONG nnd propagate it back-along the marked chains to 
the original men. So the nen whose chains ended on LOWE become marked, and 
the rest do not. This is tedious, but not unreasonably so, given the 
problem, There arc Other marking strategies which would work equally well. 

By now, a picture should be emerging of how all of this eight he 
used in a recognition program. If we are faced with a creature that bus 
webbed-feet, a fiat bill, and irridsscent green feathers on its head, we 
Simply nark ftll web-footfid creatures With one hit, all Flat billed 
creatures With another, and all green feather-heads with a third, then look 
for the in t ers e ct ioh - - the set bf all nndes with three marks, if we have 
mc’-e features than marker bits we can mark a few features t intersect, mark 
the still-viable candidates with a single bit, and repeat until we're done. 

Of course, this intersection - finding is not tho whole story Of 
r ec ognition „ One man's large nnsc may be another man's medium, and: aside 
from such blurry boundaries we will often get errors and freakish 


exceptions —the famous thro*-legged caw, for example. Finding the 
intersection or properties is a very quick and fairly reliable wav of 
flitting an initial hypothesis, but we Still Hied to devote seme effort to 
evaluating and improving the fit. Also, m fields Lite vision, there most 
■ { s ™ interaction between the evolving hypothesis and the feature 
extractors. This represeiltatian is a powerful topi for a recognition 

° f th * Kleins outlined in my original working 
paper Li | still must be worked on, 

n II}* fast intersection is a part of what I meant when I said the 
aS a ni<> ^ 1 Df h " mf ' Host people, when 

IZ lowilr ! 8 rsrogn it ion task. Feel that Che hypothesis emerges in a 
sudden flail 0 , inspiration—if ii comes at all. Before this flash is a 
period OF ainless input gathering, and after it may he a period of fitting 
and weighing, but there normally seems to bp very little possibilltv-hy- 

r e * rCh ' ^ IJlUsi al1 sls *'^ failed. I submit that the sudden 
appear ante of a Single node at the intersection of a bunch of features is 
about as Close ta a flash as we are cv gr likely to get, in most other 
systems, including ny suggestinn-demon network, the finding nf a hypothesis 
ln-'Oives noth mare work than the subsequent verification, 

I dnn r t (iav e ton nuch tn say about the process of generating an 
appropriate accessing formula for a given node. It seems like a relatively 
Straightforward process, though perhaps a bit marr complex than the 
deenrkr-j procedure In a way, it U like giving directions to a motorist: 
llrst you tell h.im tD go to scrap mutually-known landmark near the 
destination—in this case a node with a none that he knnws--unc! then you 
givo him the series of street names corresponding to the best path fron the 
landmark to the goal. This is something of am art; the best path is the 
one most easiiy followed, not necessarily the shortest. Yon might describe 
--ttain intersections Along the way, sei that the traveler can verify bis 
progress. Thus we might have "the length af the nose of flax's 
grandfather's yattnp Autrlrfir mistress.* The adjectives may provide the 
listener with new information, but if lie krnjws (or knows of) the lady, they 
will Instead provide a redundancy useFui Tor checking whether he is 
following the path Properly, 


Obviously, if the person creating the formula is to be so 
considerate of the listener's needs, he must have a good model Of the 
listener s Information network. The words and the nodes used to anchor the 
Formula, must h*vo a meaning for the listener as well as for the Speaker, 
Speaking to ny brother I might say "Champ's mother", but to a Stranger I 
have to say ’the nathcr OF the German Shepherd my family used to 

. 1 must tllvide m y knowledge into packages on the basis of wlio 

might know Bhy giver itesn-ivaryon*, just ny family, all scientists, all 

?ho«\tS dfllltS 't Wltf .: SD 0If "" aild 1 ™*t operate within the proper subset of 
these packages when framing an utterance for some particular audience. We 
will see a possible implementation for this in the section nn contexts Of 
course. I have just scratched the surface of til* formula-generation 
problem; this looks like an interesting thesis far somebody^ 


12, Handles and IK'Links 


Id the next rew sections I wilt describe ft number Of embellishments 
to tbs standard GF-link. These new nechanisns are H in general. not 
designed to represent new properties of real-world objects. Rather* they 
implement what, in ny previous working paper, 1 called! the tfirtif fdrp 
1 xt_/o rntrt t i OP in a description: statements about how the descriptive 
information is to be interpreted end used. Some of these new mechanisms 
may so^m tn be weakly motivated at first, but they will he very useful in 
certain applications we will encounter later. 

Before proceeding, I should point out that once links have reached 
a certain level of complexity it freCQfflOS possible to Simulate some of these 
mechanisms. For irtStunre, instead of introducing ft four -pronged link, it 
might D5 possible to got the same efFect using a pair of three-pronged 
links. In deciding what new link-types to present.here, I have tried to 
optimize for ease Of understanding* introducing new nechanisns whore they 
seemed to represent natural and often-USCd relations, hut hot for 
particular ijuirky cases. The boundary is largely a matter of taste- In a 
hardware implementation the boundary would be ft matter of economics, and it 
might fall lit a very different place, 

The first of these new mechanisms is something I tell the handle 
node of a link. Each link cones factory-equipped with a full-fledged node 
of its own. This node Can either be thought of ft£ an integral piirt of the 
link-unit or as being attached to a fourth prong--the effect is the seme in 
either case. 

This handle node represents the itew o/ irr/Dmirtior? stated by the 
attached Link, Statements about this piece of Information can now he 
represented by simply attaching the ftppropriftte links to the handle node. 
About a, statement X, we can now say n X is ft LIE" or ''WohLG-ALrlAWAC is the 
SQLIitCE nf r, It is- now a simple matter to label the essential and 
incidental links of a mandatory nods. 111 short, the handle gives ns a way 
of getting hold of the associated link--hence the nemo. 

Note that not every link cen hftve something said about It--this 
would lead to an infinite profusion oF links. We can not even afford to 
give every handle node an identity link--fln exception to the rule stated in 
Section S , Perhaps every handle-node, by virtue of being a hen til e -Ti 0 d e „ 
should be considered to have an impiim link -stating "X Is a LINK", but 1 
doubt that this would make much practical difference. 

Ve might he tempted to Say that handle nodes are optional, that 
they are only attached to those links that we have something to say about. 
This is impossible, however, because the handle nodes serve another purpose 
as wollr they provide us with a way of narking sets of links. At any 
given instant, we may want only sine subset of our knowledge to be active^ 
the inactive links Should play no part in any computation - This allows us* 
for example, to have multiple inconsistent world models represented In the 


sane knowledge net, but Using difr-irent sets of lints. By marking ■one bit 
In the hand Is nodes of the active links, and- by ordering that lints not so 
marked ignore all CPU instructions, wc can achieve the desired: effect, We 
will see ill tlse Section on contents haw such activation markers propagate. 

Actually, we will see a bit about this topic right now, as we 
consider the f.lMint [pr, sfwrpttries, the IM-pr cxtjj)', Every one of our OF- 
links li&5 yet ibother prong, running off to a node called its content. It 
is along this prnng that markers travel to activate the lihk, Normally, if 
a link's context Is activated, ths Link will be activated as well. 

The relation between a piece of information and its context is 
normally specified by the English words "in the context of" or Just "in.", 
Thus we have "ABRAHAfl is the FATHER af ISAAC trr the BIBLE" or “ .GRAVITY U 
on IWVERSE-5QLIARE-F0RCE iff NETtflN I Ail-PHYSICS", If we are discussing some 

other book or EihSteinin.n physics, these particular contexts would got he 
activated and these pieces of information would play flO part in the 
processing. There is very general context, SEVERAL-K-NOWLEUGE„ that 
contains those facts that have no place else to live. The context nodes 
themselves are tied into a tangled hierarchy hp.se3 upon the relation "is a 
sob-context of" rather than "is a kind of", but that's an issue 1 don’t 
want to get into yet, 

Note that a Frane (in the fliusfcian sense} is a hind of context, 
binding together a group of relations, Thus we hove "MARY is the flOlUER of 
CLVBE in the JOKES-FANILY" or " HURRAY is a BASSOONIST iff the BOSTON- 
SVrtPHOMY-ORCHESTRA" or "SUSAN is the GUEST-OF-HONOR at the PARTY". (Well, 
you can't Win them ail! I think that "at" replaces "in" te Flag a 
particular spatio-temporal context, but this is tricky.) 

Sometimes it is hard to know whore the boundary between OF and IN 
fails. Is the crankshaft a part of a car or a part tit a car? Both, I 
suspect. This question will he treated at length later on, as will the: 
whole issue of J>$rt-wh0le relations, In some of the Context schemes I bovg 
tried it Is Useful for a node to have many IN-prongs, In this case, we 
don't have proraps at all, but separate H'-iinAj running between the link's 
handle node and the Various context nodes.. This, too, we will cover later. 

In fact, 1 hove said all that I really can about IN-1 inks without 
opening up the whole context issue, and we aren’t quite ready for that yet. 
i Just wanted to givg you a quick look at the machinery that- exists to meet 
the context problem, sa that it won’t seen too strange when we do need it, 
and 5a that I can sneak i) few IN-links into examples in the meantime. 
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This entire descriptive system Is based upon the principle that It 
is nnrs efficient for an individual to inherit a large group of properties 
from an hpproprjate jupsrior class than to state all of an individual's 
properties directly, ftisch of this advantage is lost If the entire of 
property must be usod as is, without any modifications. The most 
valuable classes--lho.se h'Lr.fo the largest bundle Of properties^-would have 
practically no members. In the system l have described, It Is trivial to 
add new properties bet somewhat harder to cancel out inherited ones. 

The tangled hierarchy system eliminates some nf the need for a 
property-cancelling mechanism. Let’s assume that all Of the cows we have 
aver Seen are black. This color is attached to the COW node and is 
Inherited by all individual cows, Then we sec a brawn cow, Instead of 
trying to cancel the BLACK property and add BROWN for this individual wo 
can reorganise the COW node as ihown in Figure 13. Wc now have two kinds 
of cdws, black ones and brown puss. Both aro perfectly legal classes, not 
OMCeptians, The rest of the COW properties rena irt with the superior COW 
node and are inherited by both types of tmi, The exception has been 
n 0 rmo i i zed. 

Normalisation has iti place, but It is not Ehb whole story, For 
one thing, w* would be reluctant to change important high-level nodes every 
time Seme random exception comes along. Imagine, for Instance, a feathered, 
hippopotamus. Von have never thought of such a thing before, and never 
will again (if you are lucky)* Is it reasonable to suppose that for the 
sake of this Single image you have permanently divided the class of 
hippopotami into feathered and non-feathered varieties? 

Far Ahother thinj, the normalized sub-classes are too equal* Often 
™ “ BBt the s ^ slest to assume some specific default value for n property 
unloss this default is Specifically contradicted in the description Ton 
may know that albino elephants exist, but if I say ’■elephant" to you, you 
Will almost certainly think of a gray one. It is possible to label one of 
su ' h ' cl ^ Ss “ s as the typical One, and to place new class-members into 
tnu sub-class, but this becomes very clumsy if a Class has many default 
properties. 

So we still need a true exception mechanism; Some way for a given 
node to cause certain superior Links to remain silent when an access is 

nade to some property of that node. To achieve this, we have a paw link- 

type: the exception link. (Actually, *e rauld do this jab with an GF- 
UnK, but I Joe: that the separate link-type makes for a clearer 
presentation ) The exception Link runs fren the node representing the 
exceptional individual to the handle node of the link that is to bo 

cancelled. Figure 14, for instance, tells Us that Clyde is a unit to 

elephant, 

hljenever we are about to begin a ladder sweep to find song property 
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Of CLVDE, Wfi first send out markers ever CUPELS exception Links to set 0 
cancellation bit in the hand lei of the cancelled superiors (or perhaps to 
clear at activation hit). Then (hfi sweep proceeds as usual, except that 
the CPU Commands are addressed only to (Ion-cancelled links. Note that the 
cance. la tion signal must reach- a link ijc.fare any barkers have crossed it; 
it would be vary hard to track down and neutralise *U of the propagating 
descendants of an Illegally-passed marker, 


In fact, the above picture is uver-sittplified, Ah exception Can he 

.'-■.hr. fl jo n r. -i s '.■ - n" 'i i" . s well r.-. r.., ; . n individual nodi': . . it should he 

inherited by tha class-msnhers, just like any other property. In Figure 
!5 for example, we see the class Of albino elephants, of Which CLVDE is a 
member, Hole, too, r haC because of this system's habit of semetiraes 
expressing properties m terns of group membership, an exception link might 
have to run from a node to &rie of its own superior 15-A links. This looks 
dangerous.; w e have exception: markers cutting off the path to superior 
nodes, and superior nodes--in mi d-sweep--gene rating- their own except ion 
markers. It would appear that the only way to avoid potentially chaotic 
races netweon markers is to proceed carefully, step by step. First the 
exception markers are sent out; then, over the stlll’-valid upward links 
the first layer of superiors is narked; then, the exception markers □□ out 
rrom these nodes; then the next layer of superiors is narked;, and SO on, 
Tois is orderly, but now an upward sweep takes twice as long. 

Can Che exception and superior marking steps be done 
simultaneously? [ think so., since a node would have no reason to cut its 
awn immediateEy superior I$-A links-any more than It would have reason to 
Cancel its own propcrties--but only those linka ohoi'e one of its superiors. 
Thus, from any given level, hr would be sure that the exception marker 
would ranch a link in the current step, and the regular markers would find 
t.Jiv bridge already burnt Khan they arrive one cycle later. There is Still 
potential for trouble if there are redundant paths of different lengths to 
<l superior nod 0, but this is a perverse situation and I suspect that It can 
be deaLt With by forbidding such redundancies where possible and by adding 
kludges where necessary, 


Given an efficient exception tiechanism, • w* can afford to mate 
liberal use or defaeH assignments, with all the advantages Kin sky bos 
Claimed for them, and exemplars of the sort I described in my previous 
paper The vario-us links id a description can be labeled according to how 
reluctant the system Should: be to take exception to each, Knne may be 
sacrosanct: no individual may alter such a node. Others may he mere 
deTault conditions, Such as the rad color of an individual's image of FALL,; 
these can be changed with complete impunity. Such properties are, of 
course, attached to the handle nodes of the links in question These 
discresmnetf values, play an important role in recognition: The no re 
exceptions an individual Mist make to fit into a particular category-- 
■especiaiiy importsrrt exemptions—the greater Is the likelihood that the 
individual would be happier somewhere else. 


14. limed late Connection 

There are really two distinct senses ttl which properties and 
rmJfribersh 1 |>S 0-fln be assigned to a node . The first is the sense we have seen 
in- all or the enamples so far: the properties and memberships attached to 
a node are meant to apply to all of the inferiors of that node. In 
addition to this* however, we need a way of sayloy things about the- node 
I £s el/--about its characteristics ftS n group or about the way it functions 
as a role in a larger description, for somn reason, this distinction was a 
very hard one for m to grasp: it was the first of these auxiliary 
mechanisms to be developed and it Caused me a great deal OT trouble until 1 
figured out what was poing on. 

Consideri far aijstaoco, the Statement that a person has two Eyes, 

We night represent this as shown In figure 16. (The wiggly lino Is an tN- 
llfils.,) Bdt if Che number property were connected to the EYE node in the 
usual way, each individual eye would Inherit the property of twoness, This 
IS clearly not what- we intended by this statement. The twoness refers to 
the EYE node itself-specifIthUy, to hew that node is to bo filled in the 
context of a particular person--and not to any member of the class EYE. 

To represent this distinction, we create a new hind of 1ink~to-node 
connection., Which 1 will call fittereUitfic DffRjTEctian. In contrast to the old 
kind oT connection, which is meant to ho inherited by subordinate nodes, 
lEme-diat'e connection indicates that the property or m-emhership is meant to 
apply only to the node in irmsdiate contact with the link, in that node’s 
capacity ns a group or role, I pro For to thint of those contract fore smses 
as being a property of the connection It SC If--perhaps the link unit prong 
is attached to a different terminal of the node Ufl 11 - - bu t It Could just as 
easily be in "inmediacy- bit 1 " associated with the prongs or even a -distinct 
family of link units. In the diagrams I will represent limeillate 
attachment hy a double Chevron near the point of link-node contact* as in 
figure LEu 

It is up to the various -accessing programs of the CPU to treat the 
two connection senses differently, During ladder sweeps and other 
property-accessing operations* only the normal Connections are of interest. 
The immediate Connections are consulted during the various digestive 
processes of the system When, for instance, the possibility of adding 
another EVE nude to the CLYDE description is being weighed. This is net to 
say that wo can nei?er use a parallel sweep to find immudlatt properties: 
wt might, for instance* want tp list all the parts of an elephant that coma 
in pairs. But the inmodlate Connect ions stay out of the way in normal 
property-fInding sweeps. 

Immediate connections can be used to represent many things. As wo 
have seen, they provide a way of indicating how many players h role may 
have in thy given description. Similarly, w« can label certain roles ns 
being optional, such as a moustache on a nan's face- Some of the 
mechanisms wu have mentioned earlier, such as the labelling of certain. 
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noctfls as mandatory or exclusive or the statement of a node's reciprocal ara 
Also Cftses of immediate attacfiiifriu. A group may be labelled as empty or a 
member of a group may be labelled as typical. fl nude may he tapped as 
representing a group or an individual in a similar nannor. 

;Ms whole business of individuals is more subtle than it might 
appear. I was tempted at fir^t to make all of an individual's properties 
Connect immediately to the individual's node,, After all, there can be no 
question of these properties being inherited by an inferior node, fer 
individual nodes have no inferiors. ^The different appearances of an 
individual At different tines could be treated as inferiors of that 
individual's node, but I have found the context system to be more useful ■ 
for representing the changing properties of an individual.) 

But we can change Ah individual node to a class with surprising 
ease, as when we refer to someone- us a Hitler Or a George Washington. The 
boundary can often he very blurry. Is Lassie a dog or a kind of dog? 1 
understand that there were many dog-actors who played this role over the 
years. fl may be a specific individual transistor in a circuit diagram, 
but if that circuit is manufactured, each unit will have its own version of 
Tl r and some of these may have exceptional properties. So the 
individuality of a nodE may change from context to context. 

This being the case, I decided that to impress individuality upon 
hundreds Of incoming property links was a mistake. It is no harder to use 
normal connections on individual nodes, so that to create, is metaphor the 
system morely has to hook in an Inferior node and Create an exception to 
the Single l W.J. v 1 DUAL property of the superior. I do not yet have a good 
answer for the question Of how we know, for example* that when I call 
someone a Hitler E am referring to his personality and political 
philosophy, rather than to his blood type or place of birth* Metaphor i«i a 
complex mental operation. 

There may be a few aero lists fer immediate connections that 1 
haven't really worked out yet. Consider the statements "a resistor is ’a 
component in efectrooicj- and "a crankshaft is 0 component U p ^ r ". If 
wg have a specific resistor we are still in electronics, but if we have a 
specific crankshaft we are in the context of some particular ear. In one 
case we create an Instance and in the other Case we don't. Perhaps this 
difference can bd flagged by connecting the IN-link immediately to 
ELECTRONICS! and normally to CAtt, but the context system is still ton 
noddled, for me to bfr sure of tftis. 


lb. -Node VJ :■ me* s- a lift Speech ftecogn i t i on 


At this point ih the presentation it would be nice to pull 
everything together and demonstrate its operation in a few extensive 
examples or tUlte description fraffles-those that dp not have a mv<i for 
multiple contests to represent tine-varying features, Examples of this 
type night include the description of an electronic network, a wab of 
family re lation ships, or the description of a ctlsen.se in terms of its 
symptoms. Unfortunately, i ria not have any Vary complex examples worked 
out yet. The tools I have described SO far are Still new, there are still 
soma plenas missing {minor 0H3S, I think), and the unsettled state of the 
context system keeps sending tremors through everything else, since it is 
tied to the format of the basic link. Once the context ness la 
Straightened out, the development of such examples will be my first 
priority, 

1 would, however, like to present one example that Is at least 
partially developed: the mechanic for associating nartes with nndos. This 
will invuive iom hand-waving, but hopefully not an intolerable amount. 

The first point to be made is that the node representing the 
concept CQV and the node representing the norrf fl CPW n are distinct, though 
linked together, A CCW has four legs and eats grass; H CW" has three 
letters and rhymes with 11 how', COk 15-A N/UfflAL; -COM” rs-ft NOUN, "CW 1 * is 

the NAllE Of cow. A WANE Is-a WORD. Cow Is the IfEAWING of " cow. And SO 
on *. 


A concept node may have many name nodes dr none at all. We have 
seen how accessing [omulas can be used to reach unnamed nodes or those 
whose name might net bo shared'by both the speaker and the listener, I 
would guess that less than Uii percent Of the nodes in a person's head havo 
nilHS£--asSiimihj| 1 of course, that people Jurae nodes in their heads. Unless 
It is a nonsense word, every w 0 rd has at least one moaning; some have many, 
he will sec later how the context system can be used to eliminate this 
ambiguity in mast cases, and also haw multiple languages can be handled. 


We must attach tc each Ward node a Spelling and a prgpunciation-'- 
ont a sequence of letters, the other a sequence ofphonews, Thorp are at 
least two possible ways to represent such sequences. One way is to use ft 
master WORD frame with perhaps twenty role nodes representing absolute 

!™ r i; D !f U0 " B M , ^ T :i ,ETTER ' and so on, The spelling of 

. ea&ll y indicated by filling these roles: C is the FIRST- 

Hrnuh R Cow 'f c is the SECOND-LETTER df "COW", W is the THIRD-LETTER of 
, J T _j l j Pronunciation could be stored by a similar method. Very long 
words could be broken lip and stored as a sequence nf sequences. 


Such a structure may havg its uses. but I don’t really believe that 
people use ahsolute-position schemes for storing Spellings or 
pronunciations {except, perhaps for a few crossword-puzzl*. fiends and 
Scrabble hustlers). If we did, it would be a sirapla matter for us to list 


every worrt w* know that fits the patter* -AT-W, Simply by Intersecting 
the sets Of words with these particular letters in the specified positions. 
Any word^node that collects all four marks is a winner. {Did you think of 

. „I €ry . K tn most I^ple cannot even tell you the Seventh letter 

in Mississippi' without Countin a, 

AH of this suggests that these sequences night instead be Stored 
list structure, with a specified first member and a string of other 
jrs linked by SUCCESSOR-QF relations (see figure IT). Note that the 


in a 
member 

individual letters in N OOW' are not the alphabet letters themselves, blit 
tnjionces of them, this makes it possible tP speak Of 
Mississippi" as a distinct entity. 


the 
i, 

"the second £ in 


The list structured sequence has same important advantages in 
representing phoneme sequences. Speech Is an Irregular Irconiiiu stream, 
and a burst of noise migrht cover an unknown number of phonemes. This sort 
of tiung would wreak havoc upon a system tied to absolute letter positions 
3ho list structured system, On the Other hand. Could pick Up the next 
intelligible sequence t locate all words containing that sequence ropordlesj 
Of its position mithirr the word, and proceed from there. 

Consider how the system might recognize the word "today" as It is 
usually mumbled. The incoming phoneme sequence might look like this j T- 
sounrlr some nondescript vowelr T or D sound; long A-sound. The CFU orders 
every word node to mark its first phoneme earic If that node Is a T-sound 
.he mark Is passed to the second node if that node is any member of ttia 
vowel class; otherwise the nark is erased. Then on to tho T or D, and 
finally on to the long A. If this is not the end of the word, the mark is 
killed anyway. Probably only one mark will have survived the journey, 
especially since we may have pre-selected the original Sfit Of words 
according to part of speech or subject area. The mark is then sent back to 
the word that spawned it, and we have a winner. 

Id effect, the string of phonemes attached to each word-node Is 
acting as a finite-state acceptor for the incoming strean, and all of these 
acceptors arc acting in parallel. If the governing CPU program is smart 
enough. these acceptors can be quite flexfhle In dealing with such thlhgs 
as indistinct ward boundaries. -One set Of marks can continue Oh with the 
Did set of words, while another set Starts froti the beginninn of what night 
„! * nBK “ 0r ^ &Qth can be Stepped along at the same time until cue 

" Che Other riles put, This does not necessarily happen in strict real¬ 
time: the intoning sequence is almost certainly buffered SO that tho systen 
«b recover Trom bed siyLis i Dns.--on over Patous pruning of the initial 
possibility set, ior instance, Obviously, it will take a lot of work to 
iBHloment those vague ideas, but this system does seem to r.ake the task 
lucre manageable. 

It Is fun to Speculate about this Sort of system, Perhaps, for 
instance, the concept nudes lave IP one part oT the brain and the word 
nodes live in another, {Remember my earlier speculation about the Use of 
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neighbor ho otis to save crossbar capacity?) Suppose, too, that tine supply of 
direct connections between the two areas is inadequate for modern needs , 
that it gets used up in early childhood, Et might then be very hard tp 
form a new qaw‘■waning association nuichly, and the associations that are 
formed might be very indirect and unreliable, like a phone qonnectipn from 
Boston to tlow Yor* by Wfl.y of AlttsSta. This might explain the plight of 
people like me who can remember both names and laces, but have a terrible 
time keoping the two tied together. The List-structured form I have 
suggested for words might also explain why, when we can’t quite remember a 
word, It usually comes bark if we can get the first letter. The second 
letter, unless it is very unusual, gives us much less help. Such ideas are 
not to Pc taken seriously, however. 


16 . The Content Hierarchy 


The word "context 1 . ns I h-Eive been us. lug It Id this paper, refers 
to a number of related topics. First, there is what l caU ohjolute 
co rr d cx 1 1 pieces of ihformation (links) can he valid lfi some contents, and 
completely inoperative m others. This is similar to the use of "contexts' 
in CQNHIVER [ID] to allow multiple inconsistent warIct-modols to co-CHlst in 
the same riata-Da.se. Then, there is the use Of contexts as fldJHtols, In 
any given context we will expect to find certain things, arid! these 
expectations can be an inportSht tool in the disambiguation of inputs. In 
a Zoo, one is predisposed to see lions, tigers, and elephants; on a. farm, 
one expects cows, pigs, ^nd chickens. Finally, there Is the use of some 
Sort of context-relatad grouping mechanism to create complex frame-like 
structures Of wholes, parti. Sub-parts, and ail of the relations that tie 
thole parts together. 

Taken as a whole, the context package is by far the least-developed 
part of the system. At present, I have some ideas about how each of the 
above-mentioned context sFleets tan be achieved separately, tuft [ have not 
yet Found a way of pulling everything together lota a coherent system with 
well-defined areas of responsibility. | don't think that E am too far frnn 
such a synthesis, but it did not seem wise to further delay this paper 
While i work on this. -So what we will sac ih this and the following 
sections are a collection of fragments that look to me like pieces of the 
answer t hut that do not fit together very well, They should, at least, 
serve to illustrate what kinds of things need to be done in this area, if 
not exactly the right ways of doing them. 

We will begin by considertog the absolute Context'system, &y 
"absolute". T mean that in a given context .a fact is either present or 
completely absent—never anything in between. This is different from the 
habitat system, which pushes certain items into the foreground, but which 
never completely rules out any jiossibii ity. as we will see, both systems 
U^e the same context hierarchy, but Ih different ways. 

Ue have already seen raosJt of the machinery that we will need for 
implementing absolute contexts, He hove seen how, by setting or clearing 
certain bits in the handle-budE of a link, we can label that link as active 
or inactive. He have seen that the CPU can Specify, in its CoranandS. to the 
iln'k units, that only those links marked as bclhg active should respond. 

We have seen how activation markers can he passed along Ill-links from a 
context nude to the various links that reside in that context. What we 
have not seen i5? anything about the naturE of thus* context nodes or About 
hew an appropriate set of them can he activated at once, 

I think that we want to use the same modes to represent both 
conceptual entities and Contexts, ConsidEr a node like tFXAS. It is a 
perfectly good conceptual node—a STftTE of (and id) the USA—with an 
appropriate set oF properties; or. area, a population, a capital, andi so on. 
Yet TLKAS also functions as a context; in TLXAS certain laws are in effect, 


and certain other facts, are true ^Mth are not true everywhere. It would,, 
or course, be possible to create distinct (but connected) nodes for TEXAS- 
the-thinf] anf i TEXAS-the-Conlext, blit I don't at present see any advantage 
in this. As long as the CPU program is able to k£ 9 p track of which marker 
bits it is using to tag active contests, and which are being used to trace 
out properties, there should he little danger of confusion. Td its 
superiors, inferiors, and properties, TEXAS is a thing; to the fuels 
attached to It by IN-links, TEXAS is a context. So, for now, let us assune 
that every node is potentially both an entity acid a context, though some 
no.det might not make any particular uss of one or the other of these roles. 

Context-nodes can represent many types of grouping. Some aro 
spatial; u given fact might apply in Texas, on the moan, in the desert fuel/ 
desert 1, or in fly Office. Some are temporal; a fnct may be true in 1848, 

In the Nineteenth Century, on (any) Sunday, or in the reign of Hammurahi, 
Sane contexts refer to various subject areas or to Other conceptual mini- 
worlds-, real or imaginary; a fact might be true in chess, id Newtonian 
physics, in war, in k'or avirf feace, in Greek mythology, or in Clyde's 
Opinion . Note that most of these Context? also serve as entitles, with 
their own IS* A links, proper ties, and so on. 

So we now have a large and varied set of context nodes. Each piece 

of irtforfnutioTi is hung in some context by an IN-linX, and it becomes nctivo 

or inactive according' to how we have narked the context node, inactive 
links are completely out Of the way--tliey have no effect Oft processing 
speed. 

Does this solve the problem? Not really. It will very often be 

the CASS that two contexts Will differ in a few facts or a few do son, but 

will have many thousands of facts in common, This li especially true in 
the representation of the sequential stages in 4 narrative or in u plan 
dcvelopud by a prcblen-sol ving system. (5ee [. 3 J far examples.) It would 
obviously be wasteful to re-copy each of these shared facts Into every 
context that uses it, or even tg encumber each fact with multiple 1N-1 inks♦ 
It is much better to Include the shared facts in a single large super- 
context, to which the sharing contexts can be attached as ssii-corateits . 

Each tub-context inherits *11 of the facts in the super-context, except 
those which It specifically cancels; the sub-context may, of course, add 
new local Tacts of its awn. (This Idea is descended from the COMMIVf.lt data 
base [10], by way of ny own packet system [■)],} 

TEXAS-, then,, is a sub-cantext of tfSA; DALLAS, in turn, is a sub- 
context of TEXAS- Anything generally true in the !JSA--that women cun vote, 
for instance, or that slavery is illegal--is also true in TEXAS, unless 
there is specific lhforraatian to the contrary. Note that this inheritance 
af facts from a context ta its sab-context is not the same as the 
inheritance of properties along 15—A links; the CAPITAL oj the USA is 

Washingtone the capital qj texas is Austin. Like the is-a relation, the 

sub-context relition is both directional and transitive. 


All of this suggests that the sub-ccmEext relation forms Its own 
hierarchy using hardware S-Ub-OOintexl (5UBCUN) lints. Hi is hierarchy is 
distinct from the I5-A hierarchy: It has its Own lints, but it Uses the 
same collection of nodes. Since we have usurped the up-down direction for 
the IS-A hierarchy, win .speak of the SllRCON hierarchy as running left 
to right, with Che more inclusive contexts to the left; USA Is to the left 
of TEXAS, while DALLAS Is to the right] 1S1Z ts to the right of NINETEENTH 
CENTURY, The system sweeps markers leftward from a node to nark. Jill of Its 
super-contexts, and rightward lo mark all of its possible sub-con texts . 

To activate a given context* then, we simply mark it US active and. 
sweep the activation markers to the leTt, marking all of that node's supar- 
conteXtS AS well, Thu links hong in at l of the marked contests are then 
activated, and this IS the Set that is used in any subsequent computation, 

Recall that under certain unusual circumstances.--for Instance, to 
rind what color cows are--we had occasion to sweep property markers. 

(foiujiwurtf from a node. Similarly, we will some limes have reason to sw«cp 
activation markers rigHtmurd from « node. Suppose, for example, we want to 
know if casino gambling is legal in the USA, The system would first 
activate the USA context normally by sweeping leftward, but would find 
nothing relevant Lo the question. Then it would mark rightward, activating 
the sub-contexts of USA, and would find the answer; legal in NEVADA,. not 
legal in other states. It stems unlikely that the. system would ever want 
to compute in this condition, since directly contradictory sets of facts 
would be active simultaneously. Me will see some other uses for rightward 
swoops later, in the section on habitats. 

it seems clear that this SlJBCON hierarchy should be tangled, Me 
might, for instance, want to create a context representing the desert in 
Texas in the surjner of some particular year, letting this now context 
inherit all of the information resident in Its disparate super-contexts. 

Me can then hang in the new context any information valid only for this 
specific combi nation Of contexts. The situation is closely analognus to 
that in the tangled IS-A hierarchy, except that resident information links 
have replaced properties as the things being inherited, 

I am not sure whether the context hierarchy ends in a Single 
leftmost node, analogous to the TFUNG node of the IS-A hierarchy. Clearly 
there are nodes for EVERYWHERE and ALWAYS, the most inclusive nodes for 
Space and tine. Probably there is a node named GENERAL (or whatever) that 
Bn com passos all subject classifications, A REALITY node might cover 
everything that the system believes to he literally true, as opposed to 
FICTION or to the 1> INC ERE- BUT-fl I STAkEN- BEL IEFS-OF -OTH ERS. But exactly how 
these nodes are related, and whether they are all sub-conloxts of an even 
sore general mode, is at present a mystery to me. I dan"t see what such n 
node would be gaud for, except to tie up loose ends. 

There are a few important points yet to be made about this new 
hierarchy. First, note that i/ >1 is considered a part 0/ f? When A ond $ 


are Created UJ eflti ties , Cftes A is a nub-context of B iiiVen both are treated 
as codtC-fts- The Rattle of fiflttySbtlrg, for instance, is a part of the 
American Civil Har; the Battle of Gettysburg context is a sub-context of 
the American Civil War context, Anything true in the war generally--tbit 
the inf mtry Ui05 musi;* le-load Ln§ rifles, say- - is also true in the battle, 
unless it is contradicted, Once again, this is not the same as the 
inheritance of properties: the duration of the battle was not four years, 
Uc will see more about the subject of parts vs, sub-contexts in the section 
on franc-building. 

The other point is that euerir /5-fl IlnA as a SVECUti (1mA, 

but that the reverse is not tree, The Battle of Gettysburg is a sub- 
context, as he 11 as a kind, of battle. Anything true in battle generally 
is also true in this particular hattle, The Rattle of Gettysburg is also a 
SIIBCON of WAR, by way of the AHEMICAN-CIVfL-HAR node and its upward IS-A 
link, ClffiSS is a sub-context uf board-games, and elephant of animals. 

Thus', when we speak, of a leftwarj marker sweep, we really mean leftward and 
upward; rightward really means rightward and downward. To put it another 
way, the SUBCON link seems tD be a weaker form of the IS-A, link; SUBtOW 
can carry context-related markers, but not those used for property search; 
15-A links can carry both types. This is a recent addition to the system, 
and I'm not sure I hai r e ic exactly right, but it is clear that sonethiug of 
this sort is needed. Are THING and GENERAL the same node? I don’t know. 

There is a lot more thinking to be done about all of this, 
especially about, bow far we can push the analogy between the 15-A hierarchy 
and the context hierarchy. Are there exclusive sets of contexts and 
con text-cl a shes? Mandatory sob-con texts? Bo tbe principles governing the 
growth nnd change of the context met resemble those governing the IS-A net? 
There must clearly be a con text-exception mechanism to cancel the 
inheritance of « Fact Tran a super-context when necessary, Vhat are the 
details or this? All of these are questions for the future, and until 
these details are worked out, the context hierarchy will continue to be 
just an image, not a coherent theory. 


] 7. Habitat and Recognition 


Id the■ previous section we srw huw Cahlext mechanisms operate to 
control certain aspects of the reasoning and daCa-retrieval processes. 
Context plays guite a different function in recognition processes. Every 
context supplies us with certain expectations about the entities that 
Inhabit it. These expectations can serve to limit o priori the set of 
possible descriptions against which an incoming stimulus may be matched;., 
in most environments, only a very small fraction of the potential set of 
descriptions wil.1 be considered. Gf course, this pre-selection is no-t 
absolute. When faced with a surprise, the system must loosen its 
■expectations, or abandon them altogether and proceed to make the 
identification on the basis of sensory information alone- hut in the 
normal course of its operation, the system will find that its expectations 
ere confirmed in the vast majority of cases, and that they can save it a 
lot of work, 

5uch a pro-select ion system is of obvious value in a recognition 
system Unit nust compere descriptions one-by-oeo to the Incoming sample, 
looking for a match- It is less obvious why pre-selection would he useful 
in a system like this, which can simu i taircDtiJ ig weigh any number of 
descriptions against the incoming set of features. But this assumes that 
the sensory processors have already assembled a sufficient set Of features 
for an sin ambiguous identification, it is far more likely that the first 
wave of infomation to arrive for an object Will Contain only a few crude 
features, and that to obtain more will require a specific request to the 
sensory system and the expenditure of more effort. Clearly, it will take 
mere and better features--and thus more sensory effort--to identify an 
object from a vary largo sot than from a very limited one- Of course, the 
regUirenonCs on the set of features Will also be influenced by haw specific 
we want the identification to be and how certain we want to be that the 
choice is right. 

50 the mechanism of context-driven prc-solectioil is what allows US- 
to go through life giving only the merest glances to the majority of the 
things around us. To do otherwise would leave much loss time for looking 
□,t the interesting or surprising things we encounter. A knee-high moving 
brown blur in a city park is a dog. h black compact B-inch blob on a desk 
is n telephono, large colorful nnving blobs in the street are cars. And 
so on. We are free to take a closer look at any of these objects if we 
want to 1 hut usually the gross features and the context arc enough to 
satisfy us. 

The resolution of ambiguity by context is even more important in 
the recognition and understanding of natural language. Consider the 
f 0 11owing two se n te n ce s: 

"‘Babe Ruth went into the locker roam to look for his favorite bat." 

‘‘Count Dr-ftCula went into the crypt to Look for his favorite bat." 


Now, it is pretty clear that Babe Ruth is seeking et piece or wood 
and Dracula a flying n-nnal. but what is tie machinery that, tells us this? 
We cannot lank more closely at the word "bat" to determine which sort of 
bat it is, nor can w« call up the author aid ask. Clearly, the issue is 
decided by context; the first part of the Babe (tutu sentence places us in 
a BASEBALL context, which is inhabited hy wooden bats, but not furry ones; 
in the VAMP IRE-STORY context, the reverse is true. In a neutral context, 
Inhabited by neither flavor of hat, we would have to guess which meaning Is 
intended nr defer Judgement until nore informetion arrives. The process of 
deciding which context to invoke is at least as interesting us that 
context's subsequent use; we will see the details of this process later. 


To see how this habitat mechanism works, it is best to consider the 
most obvious ftrni of the problem: the habitats of animals. At first 
glance* It appears that habitat is just another property. Like sire or 
color. THE-EVERGLADES is the HABITAT of ALLIGATORS, and so nn. If we are 
trying to identify Ah animal from a verbal description, we would Use the 
£ tat ducat that It Lives 111 the Everglades just as we would use, say, the 
statement that it Is green or has long pointy teeth: it is m rely another 
fact tn throw into the intersection-find!ng process. In this case, habitat 
is a particularly valuable feature, since it is relatively hard to tell 
alligators from crocodiles and caimans on physical grounds alone. (This 
assumes that wo care to be this specific.! 


Usually, of course, we do not have to be told that a creature 
inhabits the Everglades r or wherever. More comnonly, we would assume the 
hubitAt property because we happen to be steading in the Everglades when wo 
encounter the heast, or because its description appears in a novel whose 
setting is the Everglades, ("There, in the fetid water before me, wus a 
malevolent green share* flt least eight feet long, with row upon row of 
saber-like teeth. 1 reached for my Bowie knife,.."J So, while we are 
physically or mentally operating within an Everglades context, wo can nark 
all oT the Everglades inhabitants and operate only within this set while 
looking for intersections of other features, This is the pre-selection I 
described earlier. Wig can leave this 5Et of expectation markers in place 
until we are ready to move on to another context. 


The concept of habitat can be extended to include much more than 
anmats ami their territories. Desks inhabit offices, oil wells inhabit 
Texas, unicorns inhabit mythology, melons inhabit particle physics, 
biplanes inhabit World War I, the Plague inhabits Medieval Europe, and 
Julius Caesar inhabits the Banarj Empire in the First Century B.C, An 
entity may inhabit many contexts? lions. For instance/ inhabit both 
African grasslands ahd coos. Though we have not yet seen the details of 
representing actions, it should be noted that these, too, have their 
habitats: jousting in the Middle Ages, blowing out candles at a birthday 
party, harvesting drain in September. 


This view of habitat as merely another feature makes it easy to see 
how the system would handle surprises, In my previous paper. 1 described 


hnw a recognition system might fle&l with one nr two an ana Inns features in 
an otherwise- routine identification hy looking for an excuse. A enw might 
appear to be three-legged because one leg Is hidden by another, or it might 
have stepped on a land mine. Even if we can't verify these excuses, the 
fact that such nlatisihle explunations exist gives its SOHO confidence in 
Sticking tp opr judgement that the beast is a cow, as long as it Is a good 
cow in all other respects, t>f course, if there were a truly three-legged 
caw-Uke animal, we would probably never begin to doubt our Observation of 
three-leggedness. Jt is only when the set of viable identlf ice tiO'HS 
vanishes that «e begin to circle back, looking for input features that can 
profitably and plausibly be dDubtod, And since things .can so easily be 
moved, habitat is one oT the easiest features to doubt. An alligator in my 
Office 13 Still an alligator, as long as it is a good alligator in all 
other respects. 

So let us assume that I am walking into my office and there, on the 
fetid Tloor before tie. is a malevolent gruon shape, at least eight feet 
long, with row upon row of saber-like teeth, 1 begin ta intersect these 
features with the already-selected set of inhabitants of my Off Ice--Chairs, 
tables, plants, Other people, computer' terminal, paper, electronic junk, 
etc,--£kH(i find that the intersection is empty. Since I have unite a good 
view of the thing, and. Since surprises around HIT ire not uncommon, I 
decide to throw nut the expectation marker and to reintersect using only 
the sensory data, {This re-starting of tltc recognition process may be the 
source of what we call & double-Cake, 1 This tine, a clear candidate 
emerges: an alligator or one of his close relatives, 1 roach for my Bowie 
knife,., 

Note that habitat cun be died just like any nther property to group 
entitles together into classes. Thlt-S, we have FARH-ANIHALS, OFFICE- 
FURNITURE, and RENAISSANCE-PAINTERS, )iote, too, that a context can be 
created specifically to covtr the habitats of some particular thing or 
group Of things. Thus, we have grasslands, tsetse-fly country, and the Age 
of Sail. The process of creating such a context to contain a commonly- 
associated set of entities is closely analogous to the creation of an 
entity to represent a coTmarly-assciciftttd set of features. 

This habitat systen uses the same context hierarchy that we 
developed, earlier for absolute contexts. To mark all of the inhabitants of 
5 one con text--AFRICA, for instance--we would first msrk all of that node's 
SUpor-contexts and suh-contexts (in separate sweeps), then would mark thu 
inhabitants of all of these context's. Thus, in an AFRICA context, the 
recognition process would be dealing with Inhabitants Of AFRICA Itself, of 
Its various parts (ETHIOPIA, XEUVA, THE-SAHARA„ etc,), and of morn 
inclusive contexts such as OLD-UQRLD. Note that this process would Includa 
entitles that Inhabit the QI.B-tiCRLB generally, but not those that inhabit 
non-African uerfcs of the OLD-WORLD--to reach these contexts a marker would 
have to travel to the left and then to the right, which is not possible if 
the sweep is dono jirotrerly. The recognition process would be primed to sea 
lions, ostriches, and dromedary camels, hue not tIgors, rhaas, or Eactrlan 


camels - A general context such as AFRICA doesn't Unit the set of possible 
entities very Strongly, but it nlll help to eliminate many of the potential 
confusions and ambsguities. 

Ry choosing a context at an appropriate level of generality, wc can 
fncus the system's attention as finely or as diffusely as we desire. If 
the content is BOARD-GAULS, the system will easily interpret any reference 
to such items as Bo ar r!w.ri 1 k , Park Place, pawn, or rook, but th-e term "King* 1 
is ambiguous; it could mean either a chess king or a checkers king, (i we 
move into the more Specific CHE-SS cohioxt, this ambiguity disappears, hut 
now any reference to Boardwalk, is a nera'^ecruUirr which can only be 
understood by popping up to a more general context or by abandoning the 
expectation mechanism altogether. It is not always clear which of the 
superior contents one should move to in such a cose: the raference could 
be to an actual physical boardwalk, after all. 

So by moving from node to node in the context hierarchy,, the system 
can activate whatever set of expectations is appropriate for a glvon 
Situation, including What is needed, hut excluding everything else to 
minimize the potential for ambiguity. This Seers rrJch Smoother tllhlii the 
older view of switching between a few discrete mini-worlds that have little 
or nothing in common. 

■He are still faced with the problem of choosing which contest to 
activate at a given time. Often, of course, the context Is supplied. in 
natural language, for instance, we have such con text -specifylug phrases as 
"in natural language". At the start of a movie, the screen may light Up 
with the words "Parts, 1?&9 M or whatever. In real life we generally have a 
pretty good idea of where we arE in space and time, unless we have been 
unconscious or lost. In a conversation, we may have what amounts to a 
push-down stack of the contexts w* ha vs passed through. If we are speaking 
of games, and then move either tg the more specific CHESS context or to 
some completely unrelated interruption, we will probably keep a record of 
the earlier context; any local woa-setfuttur may be an attempt to pop back 
Into the GAjIES context, 

!5ut there- rcraains ft large set of cases in which the context is not 
supplied, but must be recognized: we join 5 conversation in the middle: 
the movie begins with pictures rather than titles; we walk into a strange 
room not knowing What sort of room tu expect: we are shown a photograph 
with no caption; and so on. The "features" used in this context- 
recognition are the things we find in the context. Though 1 haven't yet 
worked out tile do tails, the general idea is to find the most specific 
context node that contains all of the items mentioned. This can he done by 
narking to the left from the habitat of each item and then looking for tho 
rightmost point at which all of these trees intersect. 

Thus, a mention of base, diamond, pitcher, hat, and hall clearly 
invokes the BASEBALL context, though each of these words taken in isolation 
has other possible meanings, Hose and acid invokes chemistry. Ease and 


collector invokes electronics, FitCfter, cep, and fork invokes table 
settings. Fork, bishop, and check invokes chess. Check, bill, and 
collector invokes finance. Bill and wing invokes birds. Vina dhd base 
invokes the Air Force. Rn it goes. The pieces of itiFcirhlfl t ion hung in a. 
context also have a role to play in contest recognition: If Eisenhower is 
President, this must be the ifl-SR's. E think that this process bf 
intersect tog inhabitants to select a context captures the essence of what 
Quill ten was trying to get at, with perhaps slightly nore precisioiif the 
use of ramlc-ts assoc tat ions between certain items 'to create expectations 
that will be useful in resolving ambiguities. 


There Is an Interesting circularity at. work hero: Wo are using the 
context to help us recognise the things in it, while using the things to 
select the context. Which canes first? The answer, 1 th-ink, is that both 
emerge together, like the solutions to a set of sinuitancous equations or 
the iunctlon-lahels in the work of David Waltz [19J, Consider the process 
of recognizing a photograph of unknown objects in an unknown setting, who ra¬ 
the objects cannot be seen too Clearly, We begin by attacking the -clearest 
objects, trying to see what they are. let's say that a COW emerges. This 
suggests that we are in sene sort of fam context- This, in turn, helps us 
to recognize other objects in the picture: a pig, chickens, a tractor, two 
silos with loading machinery. The tractor and machinery help US to further 
refine the context: It is a modern farm with mixed livestock.. Once again, 
this helps us tD recognize objects: the thing an the ground near the cow¬ 
ls an. electric nil king nachine. And so on. 

The objects and the context, then, refine each Other. The 
circularity is a converging one, once w# get the first bit of the picture 
by brute force- I call this tue-/injer recegrci£iOa. One finger (or set of 
fingers) starts at the most general object nods of the system; the other 
starts at the leftmost context. Gradually, by alternating steps, the 
object fingers are moved down and the context finger to the right until wa 
have a rather specific context Inhabited by specific objects. Clearly, the 
simultaneous emergence nf a face and Its constituent parts is another 
example of the same process. Perhaps this is what the Gestalt 
psychologists were driving at. 


One big question remains: Should entities be connected to habitats 
by special hardware HA0tTAT-links? If net, then what should wo use? I'm 
not sure about this at present, fly instincts rebel at the thought oT 
creating a huge new network Of HA&ETAT-links. Perhaps we Cflh get away with 
using the identity links hung in a context to provide us with 

habitat links; IF E1SEIfflWER is PRF.STHF.fJT in the 15EC's, then EIREWHOWER 

inhabits the 1930 l s, Ha also inhibits World War It because he was a 
GENERAL there, and WE5I-P01NT of a certain era as a STUDENT, Inhabitants 

With no Other role to play In a context can be simply iRHABiTANTS. It is 

clear that this approach would save 4 lot of links, but I don't yet know 
how far wo tfth push it. For now, we had better play it safe and think in 
terms of explicit HADITAT links. 


10, Relations and Frames 


Ve have seert some ways of connecting properties to objects. »nd of 
hanging facts and entities In the appropriate contexts* but how do we 
handle the representation of relations Like CHEATER-THAW, in which neither 
or the objects beihg related if a property hr a part of the -other? And how 
do wo handle complex frame-like descriptions which impose not only a sot of 
requirements and properties upon the role-players, but also relations 
bet we pm thnm? How do we Randle multiple! descriptions? tihat does all of 
this have to do with contexts* and whet changes does it suggest in the Fern 
of the basic OF-link? These are questions that I have only begun to 
explore, and this section wilt only contain half-baked ideas--not clear 
answers, 


Lot US bOfllh with the GREATER’THAU relation. What da we want to 
say about it? It Is a binary transitive relation with two "slots' 1 to fill. 
Both slots must be filled by entities from the class MEASURABLE-qUALITIES; 
each slot must be filled exactly unco in any particular instance of a 
CREATER-THflH relation, The first role has the name GREATER; the second* 
the name SMALLER. Beyond that there is not too much to say. This relation 
seems to be 0 primitive that must be defined hy consistent use, instead of 
by other more primitive relations. 

A possible network for GREATER-THAU is shown lp figure Ifl. The 
Structure above the dotted line is the static GREATER-THAW description; the 
structure below the lino is a particular instantiation, or this. As before, 
the double Chevrons Flag immediate connection {properties related to the 
role itself), end squiggly arrows are 1 A 1 ’-links. Since GREATER and 5HALLER 
ere singular toIes, the CPU program mist insure that each is filled only 
once (a form -of digestion when new links are added). Since these roles are 
required* the program is free to assume that the role is filled by 
5 onething appropriate, even if the player is not explicitly stated in the 
network, (Every person has a liver, etc.) The creation of the A>R node Is 
necessary because many pairs of items will be hung under GREATER-THAW, and 
it is essential to remember which GREATER is paired to a given ERALLER, 
Besides, we might want to say something about the particular relation 
itself“-that it is an nssumption* perhaps. 

Are the squiggly IK-links shown here the same type that we used to 
represent “Eisenhower was President iff 1-he IR5fl’tT I think so, but I'm not 
sure. The English COmOS out the same, but that is a dangerous path to 
follow. The uses are at least analogous; When we arc in tho OOP text of 
the A>B relation, the link from A to GREATER is In sane sense activated. 

But 1 have not flayed with this enough to be sure that I cm not missing 
soma essential distinction. Far now, let's call thorn the same. 

Though It is conceivable that we would want to reach into the 
static part of such a network to create specific exceptions, normally we 
will use it as a black box; we will only want to get at the terminal nodes 
representing the relation Itself and the slots. In this case, we need to 
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make attachments to the G REAPER- THAN , Cl RE A TEA. r and-SHALLER nodes, This 
suggests that WO can sev* ft lot of clutter hy drawing the whole GREATER- 
THAN network as a box* as shown in figure I9n. To avoid on tang lemon 1 5 ■* ws 
may have several GREATER-THAN boxes in a given diagram* but they all 
represent the sortc piece of network.* not copies* Not* that BtN'ART- 
TRAMSITI'VE-RELATION could itself he a black box with terminals FIRST-ROLE 
and SECOND-ROLE, hidden in Sid* the GREATER-THAN box (and In many other 
hoses as well}- 3f we want to get by with even less clutter* we could 
represent the whole instantiated box as a labeled arc, as In figure 19b, 
but this perhaps sweeps too much under the rug. 

Now we can define SMALLER-THAN by merely hooking up two boxes 
(figure 24))* We s]muily switch the roles of FIRST and 51 CONI? and plug into 
the terminals of GREATEK-THAN, SIIALLEk-THAN then becomes a box in its own 
right. ABOVE is defined as a binary relation between physical objects 
(PHYS0G5) with a GREATER-THAN relation, not between the players themselves* 
but between their VERTICAL-POSITION properties (figure 21)* SUPPORT can be 
defined over two physohs r one above and touching the other {Tisfura 22), 

Nate that the TOUCH box has only a single slot that can accomodate multiple 
players; It is thus automatically reflexive. 

It s**bs clear that with this sort of representation on* cam create 
rather complex descriptions* using pre-8X1Sting description hoxes as 
subroutines. A FAMILY box can he created with mandatory slots for one 
adult male, one adult Female, and many Option*! children of both sexes* 
Inside this box are smaller boxes for HUSBAND'WIFE, BROTHER - 5151 Eft, PARENT- 
OFFSPRING, etc. An ARCH box would have slots for blocks and sub-boxes for 
the spatial relations that hold between them* A SYMPHONY-ORCHESTRA box 
would have one slot for a conductor, a few for horns* and a couple of dozen 
for violins, with much internal object description, hut not many internal 
relations. An ELECTRONIC-NETWORK box is a set of CONNECTEP-TP relations 
over the terminals of elements that nay themselves be smaller electronic 
networks. As we'Will see* actions and events are represented in a form 
very similar to relations, so we can create Structures like B1RTHPAY“PARTY 
with suitably restricted slots fgr the people and things involved, a SOt of 
internal relations* and some implied actions as well, Such Structures* 
when stkull, are relations of the sari that we see in Kins ton-nets; when 
large, they look like the Nlnsklan frames. 

But there are a tremendous number of questions still to be answered 
about these structures. Vilat are the precise rules of marker propagation 
for answering the various questions we might ask of those frames? What, if 
anything* Is the difference in behavior between the IN-links used here and 
3UBCQN links? Do we really need both? How do we indicate that the two> 
bases of an arch <fa ii-ot touch? That the players in a GREATER-THAN relation 
may be any measurable quality, but that they must must be expressed in 
sans inriti? That distinct male children in a family are brothers, but that 
nobody is bis own brother? Cnee expressed, how are these restrictions 
enforced? 1 don't know, but I'm working on it. 
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It is tempting to try to express everything in terms of these 
Frame-Structures of 15-A and EN-links, doing away with explicit Of-lints 
altogether . ABRAHAH wo a Id he tin; r ;i Hi e r oT ISAAC not because oT a sped Tic 
QF-tonnettion hetuecn them, but because ABRAHAff plays the FATHER role in a 
relation in which ISAAC also plays a role (the SOM), Something like the 
EMITTER of T1 would still be hung oYF to the right since it is a osrrl oF 
Tl, but It would be bung only by a SUBCON lint, not by an OF-limk. To find 
the Jf nf V, we would mark some appropriate set of links —the IJF'-irn'iirersc of 
Y--as active, and then pick up every node hung under X by one of these 
active links, The OF-UrtivErse weuld run rightward from V and Its 
superiors, and would a)SO include the Other role links in any relation or 
Tramc in which Y plays ft part, This isn't exactly right yet. and small 
differences matter here, but it at least indicates the sort of line I act 
following at the moment. 

The point Of all this is not just -to reduce the nurber of link- 
types by one. I mentioned earlier that the OF-Unks had certaJn probJnns. 
Chief amnng those is that, in order for thE ladder sweep to work properly, 
the OF-link rungs must extend all the way down to the level or individual 
nodes. Ve cannot say that every American is the COUWTRYHAIf-OF svury other 
Anarlean; instead we mist actually create COUNTRY HAN-QF links between each 
pai r of individuals. 'Jo could, of course, have thE digestion processes 
create these links as new Americans are addfrd, but that is not the point. 

We would like to do without these links altogether. The creation of extra 
links for reciprocal pairs (HUSBAND-flf and WJFE-DF) is a less estrone form 
gt the sane problem The hope is that membership in a common relation can 
take the place of explicit low-level OF-links between the various role- 
player s in the relation. 

So far, we have had no occasion to its a the links th(:nselves--or 
rather their hand!* nodes--as contexts, but such situations do arise. To 
see this, lot’s take another look at the nature of descriptions. The- view 
I will present hero J 5 related to—though not identical to--the ideas 
expressed in the fierilh system of hoore atid Newell (13). 

A description can be viewed as having three elements: an 
assignment Statement, a set of modifications, and a set 0-f sub- 
descriptions. The assignment is JUSt ait I5-A link, CLYDE I£-A ELEPHANT, 
or whatever , Tftti modifications consist of whatever exceptions and 
additions we must make in the ELEPHANT description to- make CLYDE fit: 
siMllar ears, a broken tusk, green stripes, etc. The sub-descriptions 
consist of a mapping of CLYDi-parts intD ELEPHANT -p artS i HI is the MEAD , 

L3 is the LEFT-HIND-LEO, etc. Note that ouch of those mappings can Itself 
be a description, with its own set of modifications and sub-descriptions, 
recursively for as far as we care to go. 

In the previous sections, these modifications and sub-descriptions 
have been hung to the right of the node being described (in this case, from 
CLYDE) by OF and IN-links, but a strong case can be made ror considering 
these things as parts Of tfre description of Ctfflf, rather than of CLYDE 


itself* This is pretty much a natter of taste as long as each entity has 
only one major doscr iptiOII, or when multiple descriptions cover entirely 
different Features of the thing being described or agroe wtiora they 
overlap, It becomes a matter tif considerable importance. hnwevEr*. when the 
two views of an object are in conflict. Examples of this are the Neckor 
cubet Rill Sky'S example Of the generator in a car, which is either an 
olnctrical network or a mechanical assembly; and the voltage divider qf 
figure Z3, which can be mapped into the standard voltage divider 1m olthor 
of two distinct ways, 7tio point here is that we cap use either description 
profitably* but it would create serious conflicts to have both active pt 
once, 

it is, of course, a simple matter to create separate contacts to 
hold the various inconsistent descriptions, and to activate only pne at a 
tine. But it is very tempting to use the handle nods of the assignment 
link as the context node from which the description is hung, as shown in 
figiiro 24- This feels right, sanehow, hut I have had a lot of trouble in 
integrating this idea with the rest oT the system. It is not yet clear 
whether It is worth the effort. 

And that, AlA.3, is the present state of the context system. I 
warned ypu that it wasn't going to hang together. It does* however, look 
llk.0 something that Will crystallise nicely, given one or two morn good 
ideas-'-or perhaps the elimination of on* or two bad OitOS. 
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IS, Actions and Verba 


.■low that w® have developed the absolute context meclmnlsui for 
representing mbit ipLc states of the world-model, and a mechanism for 
binding many fads and relations lute a large frame--structure, we tun at 
last turn our attention to the representation bf events and actions, 1 
include in this category both specific events--the assassination or John, F, 
Kennedy, Tor instance —anfl event-types such as "murder" or "eating 
CniUn-hers''' the distinction is mare or less analogous to that between CLYDE 
and ELmiANT, In general, I'will use the terms "event” and ■action" 
Interchangeably; the distinction is merely that an action-frame has a slot 
for the ngent, while an event may not. I will use the tern "verb' to refer 
to the woH that names an event-type or action-type, though the relation is 
considerably nsort canp]fix than the single word-objset pairs we saw earlier- 


Stated brie-fly, an event frame Is just a relation-Frans u r the type 
tve saw in the last section, with the addition of a pair of internal context 
nodOi representing, respectively, the State of the world before the event 
and the State of the world after the event. The MURDER frame, for 
instance, would have slots for the MURDERER and the VICTIM, both restricted 
to human beings. The nURDF-RER is the agent in the frame: there is a 
CAUSES relation between him and the node representing the action All of 
the above restrictions reside in the MURDER frane itself. In the included 
ptrQRE-context m have the statement that the VICTIM is alive; tn the 
AF Efl-context the VICTIM is dead. To represent the event of CAIN murdering 
ADLL, we just create Mi instance of the MURDER Trame and plug CAIM and ABEL 
-,nt.o the appropriate slots; this event then inherits the proper relations 
and properties. 


P[ . rr ha * e a few othcr slats ar ~ "orthy of note; TIME. 

tE.Alt, and EXPANSION. Timo end place are probably self-explanatory but 
note that they are not always filled in, at least with any precise entry 
Thus, the TIME Slot for "Cain Slew Abel" would dimply be filled with PAST 
or perhaps FlIDUfAL-TIMES, though the event (if it really occurred! was 
probably of very short duration. In "Rome fell", we would have to indicate 
an interval of several centuries-not a specific; date, hour, minute find 
second. Peaces, too, say be vary specific, very general, or pot Specified 

□.L cl 1 £ „ 


ie EXPANSION slot is included in recognition of the fact that 

t0 st, - ak 311(1 niilk of •''•"is as instantaneous 
i- Sl l*lb* they In fact are made up of a Sequence of SUb-events 
Tbia sequence, if specified, 15 the EXPAN5 t-ON of the event. Tho sequence 

1S Usd tfl 3 &ther in suci * a that the AFTER-context of one 

is the BEFORE-cuntext of the next, and taken together they span the 
distance between the BEFORE and AFTER contexts of the parent event Thus 

-h^Lhh^ rt ^ ln .. Sl ™^ bel " and 1?aVB ' lt 4t tl,at < or We can 00 W to add 
1. stabbing min or by drawing his suard, placing the tip against Abel 1 * 

Chest, end pushing it in.* The expansion steps, oT course. Can have their 
own expansions, as far down as our knowledge and interest take uj- Sooner 


Of later, we will have to treat same level of events, as. primitive, but an 
action thhl IS lert primitive in one frame nay be expanded in another. 

None of this is particularly revolutionary, The frame-structures 
look like case-frames or like Schank' s networks [16]; the USC of l?Ef0Ft£ find 
AFTEffi contexts is reminiscent of systems like OPS and STRIPS; the hierarchy 
of events and their expansions looks rather like the structure of T3UILtJ- 
plaliS dr tho scenario representations of IHi 11 Wink U [1], Butt the present 
system does offer two clear advantages, not because of any unique 
structure, but because it ts able to employ the representational machinery 
that we have already developed Tor static representation: first, the 
system Is able to Specify and enforce very precise restrictions on what can 
occupy the various slots; second, the system can support a complex 
hierarchy of event-types, 

Let's consider the restriction issue first. Host case-frame 
systems use only the broadest categories to specify whet can fill a giv®n 
5lot; 4himnte object, place, jihysob, end So on. In this system, on the 
other hand, we can tie a slot into any category-node in the ndtworlt-r 
trenornl or Specific--and can enforce the restrict toil with the clash- 
detector. Thus, any two physobs tail CflfiiiiC, but only flowers can iliooifT, 
only liquids can trlcftio, only herds nf large herbivores cun stomprde. Of 
course, these restrictions cut both ways: if something is said to be 
trickling, then wr are being told that it is a liquid. 

Let me pause here to head off an obvious objection: Such 
restrictions are often deliberately violated, as when we say that a girl 
blooms into womanhood, or that sunlight trickles into the forest, or that a 
football teflrt Stampedes onto the field. These -are netaphorical uses. It 
is intended that we will notice the clash and consider the link of girl to 
flower (or whatever) to be metaphorical rather than literal. 5one 
properties—beauty and delicacy, perhaps--are inherited over this link, 
while others —photosynthesis and cellulose cell-wall s—-are not. Exactly 
how UU decide Whether a given property makes the Jump is an interesting 
question, and one which I Cannot yet answer* The relations find Other 
things in the frame are also inherited over the metaphorical link, insofar 
as possible. It is So convenient tg pick, up fi large package of relations 
in tills wny--even if they are slightly inappropriate--that such netaphors 
are used often. As the language evolves, a metaphorical usage can bee one a 
recognised second meaning of the verb with its own frame, and can some tines 
even survive after the original meaning has dropped from use, 

A related issue is the use of default assignments to fill slots. 
These are simply extra restrictions whose links beflr fi label stating that 
they may be cancelled with Impunity, as we saw in Section 13. Thus the 
SllflQT frame has slots that are restricted by default to HfK and BULLET,. but 
we are free to cancel these and substitute Other pairs of objects: BQW and 
ARROW, SLINGSHOT and F'E&BLE, etc. (This substitution 15 not entirely free, 
beca-ise the SHDOT-frasie contains 0 non-Cancellable internal relation that A 
LAUNCHES E, and the proposed slot-fillers must conform to this relation.) 


Notfl- that all of these comments About slot-rastrlction can apply to any 
relation-frame, not just to event-frames, 

h far more important aspect of the event-fraenj system is its 
ability to stack event-types Into an IS-A hierarchy. Thus, BUN and WALK, 
are special cases of TRAVEL-BY-FOOT, which is a kind of TRAVEL, which Is a 
kind or HOVE. All of the restrictions, properties, relations, end 
implications attached £q the morE general event-fraises aro inherited by the 
lower ones, and each of the lower frames adds its own restrictions ns well., 
HOVF r slrvply says that some physob is in one location BEFORE and another 
lOCfltiufl AFTER. TRAVEL inherits this, adding that the POVER is moving his 
own body. TRAVEL-EY-TODT says how this is done (that is. it adds a 
restriction to the Ek RAWS I ON slot), and RUN adds that It is done At high 
speed. Implications, too, arc inherited; if a certain radar is able to 
detect anything that mores. it certainly can detect anything that ruJti , 

But the more specific actions can also have implications, of their 
d«n:. .Jlutfiiijrff is very tiring; tfflfllfntf is less tiring. flajwiTsg away from 
the scene of n crime is more suspicious than molAfrr? away, ArcoAinp irefftf 
wjth someone is a friendly act; A renting mindoips at the American Emhassy Is 
hostile, futiflir cucumbers gives you gas; eattnff in general dots not. Note 
that not every action-frame has its owrs verb. We noud to represent the 
generalized act oT eating cucumbers so that we can state the above fact, 
but we do not need to coin a new word for this act. We do happen to ha vs a 
word--ca3in ibalism--For eating fl member of your owe Species. So it goes. 

The mechanisms responsible for all of this inheritance flro the same ones 
that allow CLYDE to inherit RAPHAL properties. 

To understand how important SUCH a hierarchy is, cohslder the 
problems ftchanfc [Id] has without it. He notices that such verbs as RUN and 
WALK have a great deal in common,, and he feels {correctly, 1 think} that it 
Is ossentisl for the rep re sen rational system to reflect these simi laritie*. 
In the hierarchical system, we simply represent these two actions as 
IhStances of a mors general action that embodies the Conmon information! 
the actions arc thus related, they inherit many of the same implications, 
but they liavE their mm Identities as wall. Lacking, such inheritance 
nathinery, Sehank's system must resort tp rewriting every instiihca of 
running or walking aj a NOTE for, as he calls it, a PTRANS)*. This does 
indeed nake evident tiie similarity between RIJN and KALE, as well the 
fact that the TIRANA implications apply, but RUN and WALK tease to exist as 
distinct conceptual entities; the differences In their meanings art 
translated into slight differences in the expansion-network that is 
attached to Che PTRAN5-in stance wtieti it is created, 

Such an approach requires a great deal of memory to represent the 
fully-expanded network for each individual action, but that is the least of 
Its problems, Sthanfc himself points out that KISS, for instance, implies 
things, that cannot 1)0 deduced from the simple fact thut two people a,ro 
touching thoir lips together. Since Che action KISS does not appear in the 
internal representation, there am Ohly two possibilities! to hang all OT 


the KlSS-implications tn the network rflp-rft5.enti.nf) each kiss as it is 
created. or to attach these implications to a complex pattern that Cftft be 
matched fl^Ainst various networks to see if they are kisses. The former has 
the probspm of all antecedent-driven systems: the question of how far to 
fJO, The latter has the problem that. In traditional systems, pattern- 
natching is a very tedious process for patterns of any complexity, For 
natching to work, the representation must he as canonical as possible--a 
given act must have only cinq possible representation, This need for 
canonical forms is, I think, what drives Schenk to sone of his more 
unpopular positions: the insistence that all actions bo expanded uttt to 
th-0 finest level of detail, and the requirement that the number of 
" primitive" Verbs appearing jn the internal representation be kept to an 
absolute minimun — fourteen or whatever the current number is. 

I mentioned earlier that verbs are the words that we use to name 
various action-type Frames, but that the mapping is more Complex than the 
simple word-meaning pairs that we found in the world of objects. I think 
that the verb itself 15 represented by a frame of some complexity, with 
slots for such syntactic satellites of the verb as the subject, the direct 
and indirect objects, and other entities tied to the verb by such -words as 
"at", "with", and "by", Whatever fills one of these slots in the verb- 
frame most fill the associated slot in the act ion.-frame. Tho Indirect 
object in the "GIVE" frame thus becomes the RECIPIENT ih the GIVE action- 
frame, and the ohject of an AT-phrase is plugged into either the TIKE, or 
the PLACE slot. As the parser fills thE slots, the clash-dotOCtor is bard 
at work rooting cut absurdities and ruling on ambiguous cases, all with Its 
usual tl 1 efh efficiency. Tfte Verb-frame would probably also indicate such 
things as the PASI-TENEE of the Verb, OF course, verbs run lit families and 
much of Che verb-frame structure can he shared. . 

] have not worked out the details of this verb-meaning mapping, nor 
thought at all about the problems of parsing and language generation ih 
this coni-ext. All that 3 am trying to do hero is to create a well-hehaved 
semantic representation that con handle the output pf a language program 
dhd Supply that program with a convenient hut powerful set of meaning- 
constraints. The rest I leave to the linguists. I do have om> weak 
conjecture, for what it tSfiy bE worth: (Perhaps the various standard 
sentence and phrase structures Cdfl be represented as scquenciS, and can 
play the same sort of parallel f ini testate-acceptor ganE that I proposed 
earlier for Speech recognition- This does not seem too far rtmovgd from 
Augmented Transition Networks [EiJ, and thu parallelism might help 
considerably. 

Let me also CPphasiae that I do not claim—yet--to have a way of 
representing eatjrutfrinsr that can be expressed in English, Drew UcDennatt, 
for OKOHple, has pointed out that the heforc-and-urter type of system has 
problems in dealing with actions iti which the ongoing process is the 
essential part. Also, 1 have not yet faetd Up to the intricacies of such 
act Eons as PREVENT or CONSIDER-, though the insights of 3chank may be 
valuable here. And even whom the machinery is fully developed, it will 


"S 1 " mt,Ch siwl«r action frames baFore we can represent 

Such AdfiA * EHfl.E7.JLt or RATIONALISE, whose description* must be based 
upon a rather extahsive sat of facts about corporate finance or psychology, 

there are a few mure loose ends to tie while w e are in the general 
area or natural Language. First, the problem of multiple Uhguage*. U 
seems c.eur that w* want to use the contest mechanism h«r«. Thu*, th° name 
of a certain concept is 'money*’ j it English, 'dinero" In Spanish, -argent" 

Lh French, bread in Early Hippie, Moot" in Hollywood Gangster, and 
funds in Bureaucratic, A sub-context hierarchy males It possible for 
dialects to Share fflQSt of thclf words-those in the super-context“-hut to 
dilfer in others. It is important to hang such words directly from the 
concept and not from the English word: had I known this In hiqb school, I 
might now be rluent in Spanish, Gf course, there will need to be a 
representation for ouch language's distinctive sat of syntax patterns As 

lli 


Another ran don point is the issue of rand i Tiers. Ve have seen how 
t ^? r9atF: <1[1 ELEPHANT node by cancelling the CRAY color link and 

adding WHITE, But huw do *9 represent ALBINO itself? I had originally 
hoped to find a Static declarative Format for such things. In the original 
proposal I suggested the Hie of a pflrcuitic packet which, when activated at 
the same time as another packet, would serve to modify that packet's 
meaning. This Idea died along with the original packet system,. It is now 
Clear to me that a concept like ALBINO rust be a procedure that tolls how 
tD create an AL&lNO-X node, given an X from a suitably restricted class, 
oometinas it will just be a matter of cancelling one linfc--a different link 
jot each item to be slhinDsd'-and adding a different on&; sonetimes it will 
be more complex, as When we apply BIG to an object. I have not worked OLft 
a representation for such procedures, but I suspect that they will look 
very much like action frames; the desired changes would appear as thE 
difference between the BEFORE and AFTER contexts. 


.Jhich brings mo to the last point, 1 mentioned that the tree of 
actions. tb*ir serial expansions, and the further expansions of these steps 
Wed suspiciously like UlilLD-pHns [3}_ Could this bo a good way of 
Stores procedural (or "how-to-> knowledge in the machine? Each action 
that is represented in the system would corns equipped with one or more 
suggested expansions explaining how Co accomplish that action, along with 
perhaps i«a information about the comparative advantages of each 
expansion, Hanning, then, would consist Of expanding the top-level action 
all the way out, until a level of truly primitive actions is reached— 

™ !scles * perhaps. This expansion would be done in the exact 
; on text of the given problem—hot in some peneralii-ed context with 

!S5r * J P™eters-- M that trouble could be spotted immediately. The 
Bi-.l-uKt or an action would serve as a statement of the action's 

Prerequis1ta c onditiuns. 


Such plans would be easy to crawl around our If an expansion 
Etoesn t pan out, simply pop up a level ar two and try a different ona.' 


And 


the system would be very good at Tinding tho appropriate action to gut from 
a current state to a goal statu: Simply do an i ntorSOCti OH/recognition to 
find the set or actions whose BEFORE and AFTER States natch the essential 
Features Of too problem. This use of recognition to find an appropriate 
operator should be much more flesibl* than the current practice Of pattern- 


matching against the goal-statement. 
for someone to pursue. 


This 1 too, ignlss Like a good thesis 



20. Concluding Remarks 


In this section I would 1 ikft to address a few of the ■global 
Ouestious that naturally arise concerning this system; Where do we go from 
here? Does this really bay* anything to do with flUrinn intelligence? And 
what does it all mean? 

The ftrst question is the easiest: The obvious next step is to try 
to resolve the remaining uneprtainties about the c(intent system and the 
flxaet format of the links. These 1 Jibes are Strongly interdependent, and 1 
expect tftflt they will all be rcSolved £ lmu 1 tame ou s 1 y--s o-on, I hope. 1 
th-lnk that [ r anally understand the problems that must be solved, but I 
have not yol had tin? to lank for a solution, I don’t really be Howe that 
the glossed-ouor parts of the system could he hiding a flaw- large enough to 
bring down the whole structure, but if there is one, it mu£t bo'lurking in 
this last major area of confusion. Everyone, including myself, will Teel 
more: confidence In the system once the Context mc&S is cleaned up. 

ATter that, it will simply be a natter of tightening down all the 
screws. A series of medium-sized descr 1 1 tion s--ol$ctroniq circuits, chunks 
cf nedteal knowledge, simple scenarios f or whnfcever*-will be worked out in 
cldtoil, and a lexicon will be developed showing how various kinds uf 
conceptual structures map into- pieces of network, Tbe procedures for data* 
access, digestion, recognition, bnd network reorganization will hav« to be- 
mad« Into precise recipes, if not into actual programs. 

It seems unlikely that any of tbe above-mentioned examples Will be 
ie big ,nat ji computer simulation will be absolutely neceS-S 4 ry, but J will 
probably bring one up anyway. It seems a simple enough task* and past 
experience suggests that it is generally easier to program something than 
to bn continually explaining why such a program would be of no particular 
value. Dne area that will require a simulation, if it is, to be studied 
properly, is the seIf -rEor ga ni2a tiop system: subtle, large-scale effects 
may be important here, 

Hopefully, the steps described above should suffice to convert the 
bag or hand-waving ideas described here into a reasonably convincing 
the £l s. If widespread skepticism persists, it may ho necessary to develop 
snrao larger and more difficult n xan e s--t he animal-description recognizer 
or something on that scale, Such things Will have to be done eventually 
in any event. Beyond all of this lies the possibility of applying this 
xnowlodge-haso as a tool in solving other problems: vision, speech, 
rtaturai language, problem-solving, children's stories, perhaps oven adults’ 
Stories. But Such things are years in the future, at loaU. 

As for the psychological reality of all this, I don't have tog aucli 
to say, I have not really had a chance to sift through all of tho 
reaction-time studies on Class-inclusion and related issues, and I do not 
, 00 k forward to the task, i'erhaps if I hesitate long enough, seine 
interested psychologist will get there first. Mo doubt the available date 


Will be both pro ruse flnrt contradictory, and n ew experiments will have to be 
done j* Let me emphasise that [ present this descriptive system merely a 
pass]hie fcfrrd of model: even if we are on the right track, there will be 
■any Changes needed CD fit the model to reality. 

A form of evidence chat ts less compelling bet - -to mc^-moro 
interesting, results from considering how wall the model predicts the 
mistakes, ragged edges, and unintentional side-effects of the human 
knowledge-base. I include in this category such things as odd patterns of 
human forgot fulness (such as forgetting a word but remembering its first 
lettor or What It rhymes With) 3 the even odder stimuli that can causo fi 
’"forgotten ' 1 face to come back: the mistakes that the disambiguation 
machinery makes, soma of which arc perceived as Jokes {What'S triad, and 
White and r&d/read all over ?)3 and the associative mental phenoraena that 
fliVO poetic images their evocative power, These things, I feel, can serve 
as the Optical illusions of the knowledge system. Any system that works 
must perform the assert Jill tasks correctly—that‘s What it means to work — 
but When two systems make the same mistakes, that points strongly to 
analogous internal structures and procedures, 

Vhilc 1 have been working on this system, 1 have from lime to time 
encountered such human mental Oddities, and have tried to explain these Ln 
terms cF ny networks. Tfle results hove been generally encouraging, but 
they are far too scattered for me to discuss at present, l hope that once 
the hugs are driven out of the context system, 1 will bo able to study this 
sort ef thing nere systematically. It is a lot more fun than doltry serious 
( number-crunching) psychology, 

ray syston does have one glaring deficiency in this respect: For 
certain types or in tar section tasks, the machine dots better than people 
de. The machine, for instance, would have no trouble in finding an 
American city whose name is also Che name Of an animal {EuffaleJ, hut nany 
people do. And the machine, in theory, would never forget anything at all, 
except deliberately. Probably much of this can be explained away as the 
difference between imaginary perfect hardware and unreliable neurons, but 
the intersect ion problem appears to be nerd complex than this. I would 
very much like to develop a theory to explain why people find certain 
111 L«rsect 1 Oft tasks to be harder than others, but I an completely at a loss 
right how. 

One argument in favor of the system is that It appears to be 
evolvable. Hy this, I mean that though the final system is rather complex, 
11 can he reached hy a series of easy steps, each of them a slight 
improvement over the previous state. The Simplest type of IS-A net is not 
tori different from- a logic net that a very simple aninal might use to 
combine a few simple Inputs with a few bits of Internal state to produce a 
Mi.-e. response. A Hbrginally more complex organism night Use such nets to 
store patterns Of stimuli that correspond to prey, enemies, and so on; a 
Simple intersection**finding program would then be needed. From this point 
the course is obvious: stimuli became complex features' and properties: the 


siJft Of the network and the complexity of the "CPU programs’ grow- and one 
by one the frills--e>!COptions r contexts, iranediate connections, and so on-- 
arE introduced. ]n short, tho system would liot have to wear full-blown 
■out of nowhere. It might, in fact, be.interesting to look at the behavior 
of various "lower' anina.ls to see whether they appear to be lacking, say, 
an exception mechanism. 


What does it mean? It’s hard te Say at present, since them Is 
still some small chanc-e that the wliglg structure will cotie crashing down. 
But if it stays up, and if it can indeed perforrt the kinds of tasks that I 
have outinned here, then 1 think jt will have demonstrated sonething very 
fundamental 2 That many of the problems that w In AJ have been battling 
against in, recent years are in fact artifices of mir overall apT>roach--&r 
our dependence upon serial Von Neumann:-type rsaohlnes and the prograpuning 
techniques that dre appropriate to then. Tl^t when attacked with the 
proper tools—with tree sweeps and parallel intersections--these problems 
fall easily or never arise. That we dn not, after all, have to resort: to 
huge aggregations of programs and demons to deal effectively with Simple 
declarative know ledge--assuming, of course, that wc have the parallel 
harttware-. I hove OO doubt that we will encounter a new set of problems-- 
perhaps worse ones-*farther on. Eut after such a 
set of roadblocks, it is encouraging to find even 


road. 


long Struggle at tho same 
a short stretch of open 


2 1. Rooti 


a v n r 11 u m; 


line living things, do not appear spontaneously in 
r-iithor, they are created by the combination and evolution of pre - ex * s t In ff 
ideas. In this final section, I wpufd like to point out ny system's' 
ancestors. I will limit nyself to previous work which has had some diract ' 
7 thinking; to list all related work would take volumes, I 
have deliberately placed this section last SO that the relation Of the 
older work to the hew cap be plainly seen. 


„ ... . Four major streams of thought which converge hero Or 
QUillian s network theory, fllhsky's frames, Winston's loaming-i 
what mJgliv be called the PU’.r.’NER-CQNNlVER traditic 


T thlnX ' Clear ln ths ^ceding suctions: F( 
QuiIlian [14 i conns the idea of using sene sort of simple parallel ban 


ire 

ig-neti, and 

ton. The contributions of 

f roiti 

net to represent things and relations, From r?inshyflz] "come s the i3« W &r* 
Ascription With default values, and a c l ear vision of how such 
descriptions can be useful. From Winston [2d] cpmes the view of 
descriptions as clysters 0 T discrete relations--as opposed, say, to 
pictures-*and the concept of the evolution of those clusters over time. 

r l 5 e ^Ak'NE.R-COk'yiVER tradition, I moan hot only the languages 
themselves [7, 17, to], but also the various attempts to represent 

SrinnS 111 Winogradts Blocks World [20 J. my own extensions of it in 

the IIIILD program [3], Chamiak's children's stnrios [2], and McDermott s 
monkeys fll], just to naag a few. CtftlfflVER was created is a reaction to 
certain shortcomings of PLANNER, 4e perceived hy its Users £15], Packets 
were an attempt to remedy certain weaknesses I had found in Conniver, 
especially lrt the arc* of demcm-eontrai - And, as we have seen this 
network theory has arisen as a result of fatal deficiencies in the packet 
sp.iI'ma . So the PLANNER-CQMNIVEft tradition was th« most direct ancestor uf 

b u t ™ cestflr ^ iLh U l«st resembles. I doubt 

ye.y ouch whether Che theory could have evolved without this chain of 
intermediate: failure-driven str-jis., 

wl .. i ' firia11 ^ brought face-to-face with the symbol mapping probiea 
Willie considering sane electronic networks that Bus small, McDermott, and 
Allen Brown were trying to represent by more conventional means. These 

n^T \ Y tt y ilT f i t iflt0 thfly forced me to go off in search 

of bother Jdaas. Sussman and Brown, it, the meantime, dev-Japed a itetwork 
epi-esenta„ion called AEflE-S which had much if, coranon with my early nets 
but without tk. spucl.l hurdwtrs. Fur . rf.il,, t h. two rtc^MtUM 

OF-link!" 0 ^ Cf 91,d AEntS had nycl1 ^ d(1 wlth the early development Of the 


need l d h Mi CaB !/ r ™ Dthcr s<furc es: My appreciation of the 

need Tor lehffted IS-A hierarchies came from Mioqgrad [2(1, 21, Z Z] and \ 

S' « f™ Hrtlidw. Fly notion of h.script inu “S i ub - 

The nif tl ° n i hAS ? e f n tBflUwlC * d b > fl ^ or9 and Newell's MER1W system £U] r 
The DVL system of Bill Martin and his group at FLI.T, [6] and the medical- 


cJi^Nnos-is system of Andes Rubin [ 15J have provided numerous thought- 
provoking examples Efenrl details. Almost everyone ht the HIT Al-Lab seems to 
he Hork]ng Oh some sort oF freme-like knowledge representation or nit 
@p nil cation using ft-ane-Uka ideas, nnd all of these systems are different. 
There has, of course, been soap cress’puj | mat-ion nf Ideas among these 
diverse approaches, Of these, the work of Ben Kuipers [B ], flitch Karcu* 
Candy Ruliwinfcle [1], and «iek Grossman [5] probably have the most in 
common with Tny own research, though thare are fundamental differences as 

Will, n-ew flcPcrmOtt and Bob Hod re have supplied much enc client and useful 
criticism. 
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